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Abstract 

As  the  largest  consumer  of  fuel  in  the  Department  of  Defense,  the  Air  Force  continually 
looks  for  new  ways  to  advocate  aircraft  fuel  efficiency.  Optimal  metrics  and  goals  are  essential 
components  to  encourage  efficient  flying.  This  research  examined  two  metrics  through 
quantitative  statistical  and  qualitative  criteria  analysis,  picked  the  most  effective  metric,  and 
utilized  Goal  Setting  Theory  (GST)  to  couple  the  metric  with  an  attainable  goal  aimed  at  making 
Aircraft  Commander’s  (AC’s)  more  fuel  efficient.  The  first  metric.  Ml  cargo  Adjusted,  uses  current 
sortie  planning  factors  and  adjusts  these  for  payload.  The  second  metric,  M2Regression,  uses 
regression  analysis  based  on  flight  time  and  cargo  to  detennine  predicted  sortie  fuel 
consumption.  It  was  detennined  that  Ml  Cargo  Adjusted  provided  a  more  robust  measure  of 
efficiency  that  would  provide  AC’s  a  locus  of  control  over  metric  results.  M 1  cargo  Adjusted  was 
then  paired  with  GST  foundational  principles  of  goal  specificity,  difficulty,  and  commitment  and 
translated  into  an  efficiency  goal  aimed  at  influencing  AC  behavior  and  optimizing  long-tenn 
efficient  fuel  use. 
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ACTIVE  DUTY  C-17  AIRCRAFT  COMMANDER  FUEL  EFFICIENCY  METRICS  AND 

GOAL  EVALUATION 

I.  Introduction 

From  fiscal  year  2014  to  fiscal  year  2015,  the  Air  Force  (AF)  experienced  a  decrease 
within  the  Operation  and  Maintenance  (O&M)  budget.  Although  the  total  force  (Active  Duty, 
Guard,  and  Reserve)  garnered  a  slight  increase  in  the  Flying  Operations  budget  ($0.3B),  it  came 
at  a  cost  to  other  O&M  components  such  as  Civilian  Pay  (-$0.2B)  ,  Installation  Support  (-$0.6B), 
Training  and  Recruiting  (-$0.2B),  and  Mobility  Forces  (-$0.1B)  (Air  Force  Financial 
Management  and  Comptroller,  2015).  This  stagnation  is  directly  related  to  sequestration  and  its 
lasting  effects.  Cost  savings  must  be  realized  in  all  possible  areas  to  mitigate  the  impact  of  cuts 
on  mission  readiness.  One  potential  area  for  cost  savings  is  aircraft  fuel  efficiency. 

This  thesis  focuses  on  active  duty,  C-17  Aircraft  Commander  (AC)  fuel  efficiency. 
Specifically,  the  researcher  compares  two  fuel  efficiency  metrics  (Reiman,  2014:52  and 
AMC/A3F,  2014:22)  to  assess  their  validity  and  applicability  in  measuring  AC  efficiency. 
Additionally,  using  goal-setting  theory  (GST)  (Locke  and  Latham:  1990),  and  the  selected  metric, 
a  fuel  efficiency  goal  is  set  for  Air  Force  (AF)  AC’s  to  be  used  in  a  follow-on  motivational 
experiment. 

For  the  purposes  of  this  paper,  Metric  1  and  Metric  2  are  the  focus  of  analysis  will  be 
referred  to  as  M 1  cargo  Adjusted  and  M2  Regression-  The  basis  for  M 1  cargo  Adjusted  is  found  in  work  by 
Reiman  (2014:52)  at  the  Air  Force  Institute  of  Technology  that  looked  at  optimal  route  planning 
and  effective  fuel  efficiency  metrics.  M2  Regression  is  used  by  Air  Mobility  Command  Fuel 
Efficiency  Office  (AMC/A3F,  2014:22)  using  regression  analysis.  The  basis  of  M2  Regression  is  to 
provide  “The  pounds  of  fuel  burned  per  flight  hour;  referred  to  as  bum  rate  efficiency” 
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(AMC/A3F,  2014:6)  in  order  to  compare  total  fiscal  year  fuel  consumption  and  efficiency 
trends. 

Background 

It  is  important  to  gain  a  macro  perspective  surrounding  the  need  for  fuel  efficiency.  This 
section  will  describe  the  current  financial  and  readiness  state  of  the  Air  Force  (AF),  the  amount 
of  fuel  consumed  in  the  AF,  and  applicable  sections  of  the  AF  Energy  Strategic  Plan.  A 
discussion  of  these  three  areas  will  provide  a  better  understanding  of  the  need  for  better  fuel 
efficiency  and  the  potential  savings  associated  with  this  research. 

The  AF  is  becoming  a  smaller,  more  cost-effective  service.  Sequestration,  voluntary 
separation  programs,  and  force  shaping  have  constantly  reminded  Airmen  and  government 
civilian  employees  of  the  constrained  fiscal  environment.  In  September  2013,  the  Chief  of  Staff 
of  the  Air  Force,  General  Welsh,  stated,  "Within  three  to  four  months,  many  of  our  flying  units 
will  be  unable  to  maintain  mission  readiness. .  .we  also  will  probably  have  to  cut  up  to  550 
aircraft,  about  9  percent  of  our  inventory"  (Pellerin:2013).  Due  to  the  length  of  sequestration, 
the  full  effect  has  yet  to  be  seen  and  could  lead  to  further  reductions. 

A  major  contributing  factor  to  this  problem  is  AF  fuel  consumption.  According  to  the  AF 
Energy  Strategic  Plan  (USAF,  2013:6),  the  AF  accounts  for  nearly  half  of  the  Department  of 
Defense  (DoD)  energy  consumption,  with  81%  of  that  being  used  in  aviation.  Further,  heavy 
aircraft  fuel  expenditures  account  for  approximately  4  billion  dollars  each  year.  A  small 
percentage  decrease  in  fuel  usage  will  contribute  significant  savings  and  help  meet  AF  priorities. 

AF  strategic  energy  priorities  support  research  pertaining  to  the  consumption  issue.  Two 
priorities  will  be  addressed  (USAF,  2013:15)  including:  1)  Reduce  Demand:  “Increase  energy 
efficiency  and  operational  efficiency  for  AF  systems  and  processes  without  losing  mission 
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capabilities”,  and  2)  Foster  an  Energy  Aware  Culture:  “Integrate  communication  efforts  using 


training  and  education  opportunities  to  increase  awareness  of  energy  impacts  to  mission.” 
Analyzing  fuel  efficiency  metrics,  determining  efficiency  goals,  and  influencing  AC’s  to  attain 
those  goals  has  the  potential  to  both  reduce  demand  and  change  the  culture  of  fuel  usage  in  the 
AF. 


Problem  Statement 

Ensuring  AF  AC’s  are  focused  on  fuel  efficiency  is  paramount  to  sustaining 
resources  and  reducing  costs.  Currently,  fuel  efficiency  metrics  are  not  utilized  to 
influence  behavior  while  flying.  Without  a  well-defined  metric  in  place  for 
efficiency,  goals  cannot  be  set  or  attained.  The  researcher  will  compare  two 
efficiency  metrics,  select  the  most  useful  one,  and  use  that  metric  to  set  an 
efficiency  goal  for  all  AF  AC ’s.  This  goal  will  be  the  cornerstone  for  a  follow-on 
experiment  assessing  the  effect  of publicly  or  privately  provided  feedback  on  AC’s 
fuel  efficiency. 

Research  Questions 

1.  From  a  metric  criteria  perspective,  is  Ml  cargo  Adjusted  or  M2  Regression  a  more  adequate  measure 
of fuel  efficiency  for  Aircraft  Commanders  (AC’s)? 

2.  From  a  metric  comparison  perspective,  how  do  Wings/Individual  AC’s  perform? 

3.  Which  metric  should  be  provided  to  AC’s  and  for  what  purpose? 

4.  Using  goal  theory  literature,  how  can  the  chosen  metric  be  presented  as  a  goal  for  AC ’s? 
What  efficiency  goal  can  be  set  for  AF  AC ’s? 

Assumptions 

Various  assumptions  must  be  made  in  order  to  properly  focus  the  research.  These 
include:  1)  fuel  tracker  data  will  be  correctly  filled  out  by  AC’s  and  entered  into  the  tracking 
system,  2)  the  effects  of  Advanced  Computer  Flight  Plan  (ACFP)  wind  and  temperature  errors 
are  normally  distributed  with  a  mean  of  zero,  3)  tail  degrade  data  in  ACFP  is  accurate,  and  4)  Air 
Traffic  Control  (ATC)  biases  placed  in  ACFP  by  flight  managers  are  accurate.  A  closer  look  at 
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these  four  assumptions  will  lay  the  foundation  for  the  research  and  provide  the  reader  an 
understanding  of  factors  impacting  the  study. 

First,  data  accuracy  and  integrity  constitute  a  key  assumption.  AC’s  are  responsible  for 
manually  tracking  fuel  consumption  and  inputting  that  data  into  the  Fuel  Data  Tracker  for 
analysis.  The  possibility  exists,  as  in  all  data,  that  the  information  is  erroneously  entered  which 
could  skew  the  comparison  results.  Although  the  assumption  of  accuracy  is  made,  outliers  will 
be  addressed  in  the  research  to  help  control  for  entry  errors. 

Next,  an  assumption  exists  that  ACFP  errors  for  wind,  temperature,  and  tail  degrade 
adjustments  are  normally  distributed  with  a  mean  of  zero.  ACFP  allows  flight  managers  to  add 
fuel  for  instances  of  uncertain  weather.  The  assumption  is  that  the  current  flight  manager 
policies  for  adding  fuel  are  adequate  and  do  not  skew  the  results. 

Finally,  ATC  biases  are  assumed  to  be  accurate.  Flight  managers  attempt  to  control  for 
ATC  issues  such  as  hold-down  fuel  at  certain  locations.  The  assumption  is  that  the  hold-down 
fuel  is  warranted  and  not  excessive.  Too  much  allotted  fuel  against  the  ACFP  plan  could 
significantly  benefit  an  AC  from  an  efficiency  perspective. 

The  aforementioned  assumptions  account  for  factors  outside  of  the  researcher’s  control 
and  give  a  framework  for  performing  the  experiment  and  analyzing  the  data.  While  all  factors 
cannot  be  controlled  and  accounted  for,  these  assumptions  set  boundaries  for  the  research.  These 
assumptions  will  be  revisited  and  reevaluated  throughout  the  study. 

limitations 

Ample  opportunity  exists  for  the  scope  to  become  too  large  in  such  a  restricted 
timeframe.  Identifying  limitations  will  ensure  the  scope  is  controlled  and  a  quality  product  is 
presented  that  can  be  utilized  for  follow-up  research. 
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The  first  limitation  is  that  AC’s  in  training  status  will  not  be  evaluated;  the  focus  will  be 
on  Special  Assigned  Airlift  Missions  (SAAM).  Due  to  the  conflicting  priorities  of  training 
requirements  and  safety  of  flight  versus  fuel  efficiency,  it  was  determined  that  training  missions 
should  be  a  separate  research  project.  A  specific,  defined  mission  set  allows  for  concise 
evaluation  of  each  metric  and  provides  a  clear  comparison. 

Another  limitation  is  that  only  active  duty  Boeing  C-17A  Globemaster  will  be  evaluated. 
It  was  chosen  because  it  has  the  highest  airlift  fleet  fuel  consumption  and  thus,  a  large  data  set. 
Additionally,  Guard  and  Reserve  Wings  will  not  be  evaluated.  Cultural  and  procedural 
differences  may  exist  that  would  be  difficult  to  standardize  with  the  active  duty  components. 

Methodology 

Statistical  analysis  comprises  the  majority  of  the  methodology  in  this  thesis.  Regression 
analysis,  comparison  of  means,  trend  analysis,  and  analysis  of  variance  are  some  of  the  methods 
that  will  be  utilized  in  detennining  the  best  metric  to  use.  Additionally,  qualitative  analysis  is 
used  based  on  effective  metric  criteria  using  weighted  decision  matrices.  Finally,  analysis  of 
GST  literature  provides  the  required  techniques  for  designing  and  selecting  a  goal. 

Follow-On  Experiment 

A  brief  and  tentative  description  of  the  follow-on  experimental  research  is  important  to 
understand  prior  to  the  literature  review  and  methodology  sections.  To  motivate  efficient  flying, 
classic  GST  provides  a  solid  framework  for  research.  An  applicable  example  of  GST  comes 
from  an  article  from  Latham  and  Locke  (1979)  in  which  the  researchers  evaluated  logging 
companies  to  find  truck-loading  efficiencies.  Despite  managerial  encouragement  to  fully  load 
the  trucks,  the  drivers  only  averaged  60%  capacity  due  to  fear  of  being  monetarily  fined  for 
being  overweight.  Management  came  to  an  agreement  with  the  union  that  the  researchers  would 
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set  a  goal  of  95%  and  post  each  driver’s  accomplishment  toward  that  goal  but  would  take  no 
actions,  positive  or  negative,  based  on  truck  loads.  They  utilized  goal-setting  methods  coupled 
with  feedback  to  influence  logging  crew  performance.  With  the  implementation  of  goal  theory, 
loads  averaged  80%  and  70%  the  first  two  months  and  then  settled  at  approximately  94%  for  the 
remainder  of  the  evaluation  period. 

A  similar  experiment  will  be  performed  on  active  duty  C-17  AC’s.  Appropriate  goals 
will  be  set  based  on  an  effective  metric  and  feedback  will  be  provided  through  multiple  methods 
(private  and  public).  The  feedback  will  provide  a  perfonnance  measurement  and  should 
influence  how  the  AC’s  perfonn  and  how  much  fuel  is  expended. 

Conclusion 

The  AF  needs  a  fuel  efficiency  metric  that  provides  a  locus  of  control  to  AC’s  that  makes 
them  believe  they  have  the  power  to  influence  the  results  of  the  metric.  Once  a  metric  is 
established  that  accomplishes  this,  goals  can  be  set  and  behavior  influenced  through  motivation. 
The  end  result  of  this  sequence  results  in  AF  AC’s  flying  more  efficient  sorties  and  ultimately 
influencing  budgetary  spending. 
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II.  Literature  Review 


Introduction 

A  significant  amount  of  previously  published  research  primarily  focuses  on  alternative 
fuels,  aircraft  modification,  cargo  compartment  utilization,  tankering  adjustments,  and  efficient 
route  planning.  This  section  will  focus  on  the  literature  that  pertains  to  fuel  efficiency  in  the  AF 
and  the  theories  that  are  addressed  and  utilized  in  the  research.  Specifically,  a  functional  review 
will  examine  strategic  energy  priorities  (DoD  and  AF),  previous  efficiency  research,  efficiency 
metric  construction,  and  GST  literature. 


Strategic  Energy  Priorities 

The  strategic  requirement  for  fuel  efficiency  can  be  found  in  DoD  and  AF  publications. 

In  the  “Defense  Budget  Priorities  and  Choices  Fiscal  Year  2014”,  energy  is  stressed  as  an 

important  investment  area  to  reduce  risk  and  cost  associated  with  consumption  (Department  of 

Defense,  2014:7).  Emphasizing  the  importance  of  efficiency,  the  publication  states: 

DoD  is  the  single  largest  consumer  of  energy  in  the  U.S.,  spending  about  $22 
billion  per  year  on  energy.  Additionally,  energy  needs  continue  to  constrain  the 
U.S.  military’s  operational  capabilities.  Large  energy  consumption  creates  long 
logistic  tails  that  are  vulnerable  to  attack.  This  energy  demand  constrains  the 
capabilities  of  our  ground,  air,  and  sea  forces  at  home  and  abroad.  For  example, 
refueling  needs  limit  the  abilities  of  our  soldiers,  range  of  our  aircraft,  and  the 
time-on-station  of  our  ships.  (Department  of  Defense,  2014:31) 

The  “Air  Force  Energy  Strategic  Plan”  discusses  the  need  for,  and  priority  of,  fuel 
efficiency  in  the  Air  Force.  Four  priorities  are  detailed  in  the  plan  including:  “improve 
resiliency,  reduce  demand,  assure  supply,  and  foster  an  energy  aware  culture.”  Although  ah  of 
the  priorities  relate  to  fuel  efficiency  in  some  manner,  the  second  priority  of  demand  reduction  is 
most  relevant  in  this  research.  Specifically,  it  sets  a  goal  to  reduce  fuel  consumption  with  an 
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objective  to  gamer  a  10%  efficiency  improvement  by  2020  (Department  of  the  Air  Force, 
2013:13). 

Previous  Fuel  Efficiency  Research 

A  significant  amount  of  research  has  been  completed  in  response  to  the  fuel  consumption 
issue  in  the  DoD  and  AF.  The  bulk  of  the  research  to  date  has  focused  on  technical  responses  to 
this  need.  While  an  in-depth  critique  will  not  be  performed  on  these  initiatives,  it  is  important  to 
give  the  reader  a  brief  synopsis  of  work  that  has  been  done  outside  of  the  behavioral  realm. 

Alternative  fuels  have  received  significant  attention  in  recent  years,  both  in  the  DoD  and 
civilian  sector.  For  example,  Heliman  and  Stratton  (2014: 1)  contrasted  various  alternative  fuels 
for  feasibility.  Daggett  et  al.  (2006: 1-8)  evaluated  the  possibility  of  replacing  or  supplementing 
current  fuels  with  synthetic  options.  Finally,  Nicholson’s  (2009: 1-61)  work  was  primarily 
focused  on  cost-effectiveness  of  replacing  petroleum-based  fuel  with  biodiesel.  Fuel  efficiency 
is  a  constant  point  of  evaluation  in  the  study  of  alternative  fuels.  The  tradeoffs  with  various 
alternative  fuels  are  important  when  determining  the  feasibility  of  replacement. 

Aircraft  Modification  is  another  area  that  has  received  attention  from  an  efficiency 
standpoint.  The  KC-135  provides  a  sound  illustration  of  this  focus  area.  Multiple  design 
changes  in  the  cargo  compartment  of  the  KC-135  over  a  50  year  period  have  left  the  aircraft  with 
a  shifting  center-of-gravity  issue  contingent  on  the  amount  of  fuel  in  the  aircraft.  In  response  to 
this  problem,  excess  fuel  was  added.  Using  additional  fuel  to  control  the  center-of-gravity  is 
extremely  inefficient  and  expensive.  Morrison  (2010)  and  McKee  (2013)  evaluated  potential 
fixes  to  this  issue  including  adding  cockpit  armor  or  specially  designing  weights  to  properly 
balance  the  aircraft  without  using  added  fuel. 
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In  addition  to  aircraft  design  changes,  operational  changes  have  also  exhibited  the 
potential  for  savings  from  an  efficiency  standpoint.  Reiman  et  al.  (2013)  from  the  Air  Force 
Institute  of  Technology  (AFIT)  examined  space  utilization  in  an  aircraft’s  cargo  compartment. 
They  evaluated  the  method  in  which  463L  cargo  pallets  were  being  utilized  and  evaluated 
whether  any  changes  could  be  made  to  the  pallet  system  to  optimize  cargo  capacity  on  aircraft. 
Ultimately,  it  was  determined  that  modifying  the  pallets  to  make  them  stackable  optimized  the 
space  and  allowed  more  effective  loading  of  cargo,  and  thus,  more  effective  flying  from  a  fuel 
efficiency  perspective. 


Metric  Development 

The  next  area  of  review  will  describe  previously  developed  fuel  efficiency  metrics. 

M 1  cargo  Adjusted  takes  existing  fuel  tracker  data  collected  at  AMC  and  transforms  the  data  into  an 
efficiency  score  through  excel  formulas.  The  metric  formulation  adjusts  fuel  consumption  as 
planned  by  ACFP  for  the  actual  payload.  Details  of  the  underlying  formulations  can  be  found  in 
Appendix  B  while  the  Excel®  methodology  to  compute  the  metric  can  be  found  in  Chapter  III. 

M2 Regression,  developed  by  AMC/A3F  (2014),  is  regression-based  fuel  efficiency  metric. 
To  obtain  a  predicted  fuel  efficiency  value,  live  independent  variables  are  used  in  the  regression 
including  cargo  weight  (lbs),  cargo  weight  squared  (lbs  ),  flight  time  (hrs),  flight  time  squared 
(hrs"),  and  cargo  weight  multiplied  by  flight  time  (lbs  X  hrs).  Cargo  weight  also  includes 
onloaded  and  offloaded  in  flight  fuel.  The  dependent  variable  is  total  fuel  consumed  (lbs).  The 
equation  that  AMC  utilizes  for  all  fuel  efficiency  regressions  is  shown  in  Equation  1(2014:22) 
and  specific  coefficient  values  are  shown  in  : 


Fuel  Consumption  =m}x  Ibsx  hrs  +  m2x  hrs2  +  m3x  lbs2  +  m4x  lbs  +m5x  hrs  +  m6 


(1) 
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Table  1:  C-17  SAAM  Regression  Coefficients  (AMC/A3F,  2014) 


Intercept 

707.94 

Weight 

67.29 

Weight  Squared 

-0.56 

Time 

16414.87 

Time  Squared 

-27.80 

Weight  *  Time 

30.13 

A  metric  criteria  model  will  be  employed  in  this  thesis  to  compare,  contrast,  and  judge 
the  metrics  to  determine  which  metric  should  be  used  for  future  motivational  research.  The  Air 
Force  Sustainment  Center  published  “The  Metrics  Handbook”  which  provides  a  comprehensive 
guide  on  evaluating  and  selecting  appropriate  metrics.  The  definition  of  metrics,  attributes  of 
effective  metrics,  and  the  metric  development  process  will  all  provide  insight  into  how  the 
researcher  will  evaluate  the  metrics  in  the  remaining  chapters  of  this  thesis. 

Beginning  with  the  definition  of  metrics,  Nowak  states  that  they  are,  “a  measurement 
made  over  time,  which  communicates  vital  information  about  the  quality  of  a  process,  activity, 
or  resource. .  .continuous  improvement  of  the  way  we  do  business”  (Nowak,  1992:2-1).  With  the 
definition  set,  the  handbook  defines  several  attributes  that  are  essential  to  effective  metrics 
(Nowak,  1992:2-1): 

1)  It  is  accepted  as  meaningful  to  the  customer. 

2)  It  tells  how  well  organizational  goals  and  objectives  are  being  met  through  processes 
and  tasks. 

3)  It  is  simple,  understandable,  logical,  and  repeatable. 

4)  It  shows  a  trend. 

5)  It  is  unambiguously  defined. 

6)  Its  data  is  economical  to  collect. 

7)  It  is  timely. 

8)  It  drives  the  appropriate  action. 

Finally,  Nowak  (1992:3-1)  describes  the  development  process  in  terms  of  a  “metric 
package.”  This  package  consists  of  the  operational  definition,  the  measurement  (data),  and 
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presentation  of  the  metric.  Using  this  package  in  an  8-step  development  process  allows  full 
evaluation  and  implementation  of  a  particular  metric. 

For  the  purposes  of  this  thesis,  the  researcher  will  evaluate  each  metric  based  on  the  Metric 
Handbook  model  (slightly  adjusted),  determine  the  most  effective  metric  for  the  purposes  of 
future  motivational  research,  and  provide  a  development  outline  to  properly  guide  future 
research  with  the  selected  metric. 


Goal-Setting  Theory 

Once  the  metrics  are  evaluated  and  the  appropriate  one  selected,  an  effective  fuel 
efficiency  goal  must  be  set.  GST  has  a  long  history  and  has  proven  effective  in  motivating 
individuals  and  teams  from  a  perfonnance  perspective.  Over  400  laboratory  studies  have  been 
performed  studying  GST : 

These  studies  showed  that  specific,  high  (hard)  goals  lead  to  a  higher  level  of  task 
perfonnance  than  do  easy  goals  or  vague,  abstract  goals  such  as  the  exhortation  to 
“do  one’s  best.”  So  long  as  a  person  is  committed  to  the  goal,  has  the  requisite 
ability  to  attain  it,  and  does  not  have  conflicting  goals,  there  is  a  positive,  linear 
relationship  between  goal  difficulty  and  task  perfonnance.  Because  goals  refer  to 
future  valued  outcomes,  the  setting  of  goals  is  first  and  foremost  a  discrepancy 
creating  process.  It  implies  discontent  with  one’s  present  condition  and  the  desire 
to  attain  an  object  or  outcome  (Locke,  E.  and  G.P.  Latham,  2006:265). 


The  background  of  GST  requires  a  look  at  the  definition  of  goals,  core  model  findings,  goal 
attributes  (content/intensity),  moderators,  goal  mechanisms,  and  feedback.  Each  of  these  is 
pertinent  to  understand  in  order  to  set  a  realistic,  attainable  goal  for  C-17  AC’s.  Beginning  with 
the  definition,  Latham  and  Locke  (2002:705)  describe  a  goal  as  “the  object  or  aim  of  an 
action. .  .to  attain  a  specific  standard  of  proficiency,  usually  within  a  specified  time  limit.”  This 
definition  is  frequently  repeated  throughout  the  literature  and  suggests  that  a  goal  focuses  an 
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individual  (or  team)  toward  a  specific  action  with  the  aim  of  achieving  a  higher  standard  or 
perfonnance  level  than  previously  attained. 

With  the  definition  set,  Figure  1  provides  an  illustration  of  the  core  model  components  of 
GST  that  will  be  discussed,  as  well  as  their  relationship  to  each  other  (Latham  and  Baldes, 
1975:123).  In  summary,  the  model  demonstrates  that  specific,  difficult  goals  will  lead  to 
perfonnance,  satisfaction,  and  commitment  in  a  cyclical  fashion  based  on  moderating  variables 
and  mechanisms.  GST  posits  that  specific,  difficult  goals  lead  to  higher  performance  results  than 
easy,  vague  goals.  “Do  your  best”  goals  contain  broad  ranges  of  acceptable  performance  leading 
to  ambiguity  in  what  a  worker  believes  is  satisfactory  (Locke,  1968:157  and  Locke  et  ah, 
2002:706). 


Figure  1:  Components  of  GST 
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Two  broad  attributes  exist  within  GST:  content  and  intensity.  Content  includes  the 
attributes  of  specificity  and  difficulty  while  intensity  is  primarily  concerned  with  goal 
commitment.  Beginning  with  specificity,  Locke  describes  it  as  being  attained  “through 
quantification  (increase  sales  by  10%)  or  enumeration  (here  is  a  list  of  tasks  to  be  accomplished). 
Thus  it  reduces  variance  in  perfonnance,  providing  the  individual  can  control  performance” 
(1996: 118).  Goal  specificity  alone  does  not  lead  to  better  results.  Variations  in  difficulty  exist 
regardless  of  specificity.  A  goal  can  be  specific  and  easily  reached  which  does  not  fit  with  the 
tenet  of  GST  that  requires  goal  difficulty  as  a  perfonnance  facilitator.  Specificity  provides 
clarity  to  the  individual  tasked  which,  in  turn,  reduces  variability  and  uncertainty  (Locke  et  ah, 
1989).  Higher  performance  levels  cannot  be  achieved  absent  specificity,  but  specificity  alone 
does  not  increase  performance. 

To  illustrate  an  example  range  of  goal  specificity,  consider  the  following  four  goals  from 
a  division  manager  of  sales  (Locke  et  ah,  1989:272): 

1 .  Improve  division  profits 

2.  Increase  division  profits 

3.  Increase  profits  by  10%  or  more 

4.  Increase  profits  by  exactly  15% 

The  desired  result  when  dealing  with  specificity  is  to  get  as  close  to  option  four  as  possible.  The 
literature  cautions  against  option  three  as  it  is  essentially  setting  a  low  goal  and  then  asking  for  a 
vague  “do  your  best”  goal  which  is  counter  to  GST.  It  is  more  optimal  to  be  precise  in  setting  a 
difficult  goal  and  allowing  the  individual  or  team  to  self-set  a  higher  goal. 

The  next  specific  content -related  attribute  involves  goal  difficulty  which  relates 
individuals  or  teams  with  a  particular  task  or  goal.  Ability  and  experience  are  key  individual 
factors  as  they  make  the  level  of  goal  difficulty  different  for  each  person.  “The  higher  the 
absolute  level  of  the  goal  the  more  difficult  it  is  for  a  person  to  achieve  it”  (Latham  and  Locke, 
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1991:214).  Figure  2  demonstrates  the  results  of  12  studies  that  examined  the  relationship 
between  goal  difficulty  and  task  performance.  The  x-axis  shows  the  probability  of  reaching  a 
goal  ranging  from  0.10  to  1.  The  x-axis  is  the  performance  output  rated  from  -6  to  6.  The  results 
show  a  positive  relationship  between  the  set  difficulty  of  a  goal  and  the  output  of  that  goal. 


Figure  2:  Goal  Difficulty  and  Task  Performance  (Locke,  1968:162) 

As  with  specificity,  Latham  and  Locke  (1991:213-214)  provide  a  simple  example  of 

setting  goal  difficulty: 


1.  Easy:  Try  to  get  5 

2.  Moderate:  Try  to  get  10 

3.  Difficult:  Try  to  get  15 

4.  Impossible:  Try  to  get  50 

Dependent  on  the  skills  and  abilities  of  a  test  group,  the  goal  difficulty  should  be  set  to  push  the 
group  to  the  highest  level  of  performance  while  simultaneously  keeping  the  goal  attainable.  One 
way  that  difficulty  is  often  set  is  though  percentiles.  For  instance,  the  easy  goal  may  contain  the 
0-25th  percentiles  while  the  difficult  goal  may  only  contain  the  90th  percentile.  The  challenge  of 
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the  researchers  is  to  determine  what  percentiles  the  different  scores  fall  into  and  make  sure  the 
goal  contains  the  appropriate  level  of  specificity  and  difficulty. 

The  second  broad  attribute  of  intensity  is  described  under  the  auspices  of  goal 

commitment.  Latham  and  Locke  describe  goal  commitment  as: 

The  degree  to  which  the  individual  is  attached  to  the  goal,  considers  it  significant 
or  important,  is  determined  to  reach  it,  and  keeps  it  in  the  face  of  setbacks  and 
obstacles.  It  must  be  stressed,  however,  that  the  feeling  of  commitment  does  not 
automatically  lead  one  to  act  in  accordance  with  it  (1991:217). 

Goal  commitment  is  unique  within  GST  because  it  acts  as  both  a  moderator  and  a  direct  causal 

factor.  If  high  commitment  is  evident,  a  moderating  effect  is  found  because  of  the  strong 

association  between  goals  and  commitment  (positive  slope).  If  commitment  is  low,  causality  is 

found  because  perfonnance  essentially  flatlines  (straight-line  slope)  regardless  of  goal  level 

(Latham  and  Locke,  1991:217-218). 

One  contenscious  area  regarding  goal  commitment  has  to  do  with  assigned  and 
participative  goals.  Depending  on  the  study,  contradictory  findings  exist  as  to  whether 
participative  goal-setting  leads  to  higher  perfonnance.  To  remedy  this  discrepancy,  a  joint  study 
was  performed  by  the  two  groups  of  researchers.  The  method  of  goal  delivery  explained  the 
contradictory  findings.  Latham  delivered  goals  in  a  supportive  manner  coupled  with  rationale 
while  Erez  was  far  more  brief  and  direct  in  goal-setting  which  caused  the  participative  setting  to 
contrast  heavily  with  the  assigned  method.  Additionally,  the  researchers  differed  in  “goal 
difficulty,  setting  personal  goals  before  treatments  were  introduced,  self-efficacy  inducing 
instructions,  and  instructions  to  reject  disliked  goals”  (Latham  et  ah,  1988:753).  Once  all  factors 
were  accounted  for,  little  difference  was  found  in  participative  and  assigned  goals.  This 
illustrates  the  importance  of  delivery  regardless  of  which  method  is  selected.  If  the  experiment  is 
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centered  on  assigned  goals,  a  supportive,  explanatory  environment  is  important  to  gamer  optimal 
results. 


The  beliefs  that  goals  are  important  (attractiveness)  and  possible  (expectancy)  are 
detenninants  of  goal  commitment.  Factors  associated  with  attractiveness  include  authoritative 
influence,  peer  influence,  publicness,  incentives  and  rewards,  internal  rewards,  punishment,  and 
valence.  The  major  factor  associated  with  the  belief  that  a  goal  is  possible  is  self-efficacy. 

Beginning  with  attractiveness,  the  researcher  will  examine  authoritative  influence,  peer 
influence,  and  publicness.  The  follow-on  experiment  will  be  absent  rewards,  punishment,  and 
tangible  incentives  (monetary,  time  off,  etc.)  so  they  will  not  be  considered  in  this  literature 
review.  First,  authoritative  influence  has  a  relationship  with  goal  commitment.  According  to 
Locke  and  Latham  (1990: 135-136),  goals  are  most  effective  when: 

1 .  The  authority  figure  is  seen  as  legitimate 

2.  Goal  assignment  conveys  (positive)  self-efficacy  infonnation 

3.  Goal  assignment  fosters  a  sense  of  achievement 

4.  The  assigned  goals  imply  opportunities  for  self-improvement 

5.  The  assigned  goals  challenge  people  to  prove  themselves 

6.  The  authority  figure 

a.  Is  physically  present 

b.  Is  supportive 

c.  Is  trustworthy 

d.  Provides  a  convincing  rationale  for  the  goal 

e.  Exerts  reasonable  pressure 

f.  Is  knowledgeable  and  likable 

The  AF  structure  provides  a  good  platfonn  to  foster  these  leadership  attributes.  The 
command/subordinate  chain-of-command  structure  enables  an  environment  where  these 
conditions  can  be  met,  thus,  enhancing  goal  commitment.  If  the  goals  are  properly  assigned  and 
“sold”  to  the  AC’s  in  the  follow-on  research,  the  authority  component  of  goal  attractiveness  will 
be  effectively  met.  Leadership  and  researcher  communication  and  involvement  are  also 
imperative  for  success. 
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Next,  peer  influence  will  be  considered,  particularly  as  it  relates  to  competition.  In  one  of 
the  experimental  treatments,  the  goals  and  progress  will  be  made  public  to  foster  competition. 
Numerous  studies  have  shown  that  competition  can  increase  perfonnance.  As  mentioned  in 
Chapter  I,  the  logging  crew  example  illustrates  the  impact  of  competition.  “Competition  was  a 
crucial  factor  in  bringing  about  goal  acceptance  and  commitment  in  this  study”  (Latham  and 
Locke,  1979:72-73). 

Finally,  a  goal  being  made  public  impacts  the  level  of  commitment.  In  an  experiment 
that  studied  college  student  performance  with  extra  credit,  Hollenbeck  et  al.  (1989:22)  found  that 
“goal  publicness  was  an  important  factor  that  enhanced  the  degree  of  goal  commitment”.  This 
aspect  of  goal  commitment  is  relatively  easy  to  attain.  Individuals  can  make  their  own  goals 
public  or  management  can  make  the  goals  public  and  facilitate  commitment. 

In  addition  to  goal  attractiveness,  expectancy  (belief  that  a  goal  is  possible)  is  important 
to  goal  commitment.  Self-efficacy  is  a  commonly  used  term  in  relation  to  expectancy  and 
“refers  to  one’s  beliefs  about  how  well  one  can  perfonn  a  task.”  Self-efficacy  is  not  only 
centered  on  effort  and  the  belief  that  increased  effort  leads  to  higher  performance.  Rather  it  is 
engrained  in  “the  individual’s  overall  or  total  judgment  of  perfonnance  capability”  (Locke  and 
Latham,  1990:115).  This  includes  effort,  planning  ability,  previous  success  or  failure,  stress 
management,  problem  solving,  and  coping  mechanisms.  An  individual  uses  some  or  all  of  these 
factors  to  establish  a  level  of  self-efficacy  for  a  particular  goal  or  task  which,  in  turn,  impacts 
how  a  person  performs  against  an  established  goal. 

Now  that  the  moderators  of  GST  have  been  described,  four  mechanisms  in  which 
performance  and  goals  are  connected  is  discussed.  The  first  method  involves  the  focus  of  effort 
with  predetermined  goal  activities,  which  causes  unwarranted  activities  to  be  ignored  and 
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important  tasks  to  receive  the  majority  of  attention.  The  second  mechanism  increases  effort  and 
energy.  With  a  goal  set,  individuals  will  try  harder  and  exert  more  energy  toward  the  desired 
task.  Third,  persistence  is  found  in  the  relationship  of  performance  and  goals;  difficult,  specific 
goals  are  met  with  increased  determination.  Finally,  action  is  attained  by  people  through 
possession  of  previous  knowledge  or  the  development  of  knowledge  (Locke  and  Latham, 
2002:706-707). 

Feedback 

The  additional  consideration  of  feedback  will  be  instrumental  in  the  follow-on 
experiment.  In  discussing  importance  of  feedback  to  goal-setting,  Locke  and  Latham  (1990:173) 
state  that  “neither  is  very  effective  in  the  absence  of  the  other.”  The  importance  of  feedback 
cannot  be  overstated  and  is  an  instrumental  piece  in  the  fulfillment  of  effective  goals.  While  the 
follow-on  research  will  examine  feedback  extensively,  the  researcher  will  provide  an  overview 
of  what  feedback  is  as  well  as  its  mediating  and  moderating  attributes. 

First,  Air  Force  members  have  a  basic  framework  for  perfonnance  feedback.  In  Air 

Force  Instruction  36-2406  “Officer  and  Enlisted  Evaluation  Systems”,  feedback  is  defined  as  “a 

private,  formal  communication  a  rater  uses  to  tell  a  ratee  what  is  expected  regarding  duty 

performance  and  how  well  the  ratee  is  meeting  those  expectations”  (AFPC/DPSID,  2013:71). 

Describing  the  importance  of  the  relationship  of  goals  and  feedback,  Locke  and  Latham  state: 

For  goals  to  be  effective,  people  need  summary  feedback  that  reveals  progress  in 
relation  to  their  goals.  If  they  do  not  know  how  they  are  doing,  it  is  difficult  or 
impossible  for  them  to  adjust  the  level  or  direction  of  their  effort  or  to  adjust  their 
performance  strategies  to  match  what  the  goal  requires  (2002,  p.  708). 

Although  the  Air  Force  definition  deals  primarily  with  private  feedback,  in  some  instances 
varying  the  types  of  feedback  proved  effective. 
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Feedback  is  unique  in  GST  because  goals  act  as  a  mediator  of  feedback  and  feedback  acts 
as  a  moderator  of  goals.  To  understand  the  mediating  effect,  the  cognitive  underpinnings  must 
first  be  explored.  Locke  and  Latham  (1990:174)  discuss  knowledge  of  results  (KR),  cognitive 
appraisals,  and  value  appraisals  to  better  understand  goals  as  a  mediator.  KR,  although  often 
used  synonymously  with  feedback,  is  slightly  different.  KR  is  taking  the  feedback  provided  and 
translating  the  feedback  into  a  deeper  understanding.  A  person  can  receive  feedback  that  doesn’t 
have  concrete  meaning  and,  thus,  does  not  influence  behavior.  The  process  for  achieving  KR  is 
to  perform  an  internal  cognitive  appraisal  which  attempts  to  provide  greater  meaning  to  the 
results  and  what  they  mean  to  the  individual.  Value  appraisals  occur  at  the  same  time  and  place 
a  good,  neutral,  or  bad  identifier  on  the  results.  Taken  together,  these  three  terms  provide  a 
mechanism  in  which  the  feedback  has  meaning  and  value.  If  the  result  of  these  processes  is  a 
lack  of  understanding,  the  feedback  is  not  properly  constructed,  and  thus,  not  having  the  desired 
impact.  As  a  mediating  relationship,  feedback  influences  goals  in  varying  ways.  Feedback 
could  cause  goals  to  increase,  decrease,  or  remain  the  same. 

The  moderating  effect  is  found  in  feedback  because  it  strengthens  the  relationship 

between  goals  and  performance.  In  an  analysis  of  feedback/goal  studies,  Locke  and  Latham 

(1990:192-193)  found  that  “Seventeen  out  of  18  studies  found  the  combination  of  goals  and 

feedback  to  be  better  than  goals  alone,  and  2 1  out  of  22  studies. .  .found  it  better  than  feedback 

alone.”  Through  a  variety  of  experimental  designs,  these  studies  show  the  impact  of  feedback  on 

perfonnance  and  goals.  The  author’s  best  described  this  relationship  as: 

“Feedback  tells  people  what  is;  goals  tell  them  what  is  desirable.  Feedback 
involves  information;  goals  involve  evaluation.  Goals  inform  individuals  as  to 
what  type  or  level  of  perfonnance  is  to  be  attained  so  that  they  can  direct  and 
evaluate  their  actions  and  efforts  accordingly.  Feedback  allows  them  to  set 
reasonable  goals  and  to  track  their  perfonnance  in  relation  to  their  goals,  so  that 
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adjustments  in  effort,  direction,  and  even  strategy  can  be  made  as  needed”  (Locke 
and  Latham,  1990:197). 

Benefits  and  Pitfalls 

Core  tenets,  moderators,  mechanisms,  and  feedback  provide  a  solid  foundation  for  what 
encompasses  GST  and  how  to  best  account  for  the  aspects  of  effective  goals.  Another  area  that 
should  be  considered  is  potential  pitfalls  in  relation  to  GST.  Latham  and  Locke  highlight  ten 
areas  of  consideration  in  their  article  titled  “Enhancing  the  Benefits  and  Overcoming  the  Pitfalls 
of  Goal  Setting.”  Each  pitfall  should  be  examined  to  determine  if  it  is  applicable  in  the  follow- 
on  research.  Table  1  provides  a  summary  description  of  the  ten  pitfalls.  With  the  exception  of 
pitfall  number  six  (monetary  incentives),  each  will  be  discussed  in  the  context  of  the  follow-on 
experiment. 


Table  2:  Ten  GST  Pitfalls  (Latham  and  Locke,  2006:334-337) 


Pitfalls 

1 

Lack  ofknowledge  or  skill  to  attain  goal 

2 

Perfonnance  goal  can  have  a  detrimental  inpact  on  group  perfonnance  if  there  is  a  conflict  among  members 

3 

Goal  is  a  threat  rather  than  a  challenge 

4 

Goals  may  have  an  adverse  elfect  on  risk  taking,  if  failure  to  attain  a  specific  high  goal  is  punished 

5 

Goal  attainment.  Past  successes  increase  goals  without  reevaluating  overall  strategy 

6 

Monetary  incentives 

7 

A  leader  tying  identity  to  the  goal 

8 

Nongoal  perfonnance  dimensions  get  ignored 

9 

Goals  increase  stress 

10 

Employees  who  reach  or  exceed  challenging  goals  may  be  assigned  future  goals  that  are  impossible  to  attain 

Pitfall  one  discusses  the  possibility  that  goal  recipients  may  not  have  the  requisite  skill  to 
attain  the  goal.  If  this  is  the  case,  the  authors  stress  the  importance  of  using  a  learning  goal  over 
a  performance  goal.  Regarding  fuel  efficiency,  a  performance  goal  may  be  to  attain  a  certain 
percentage  or  klbs  of  fuel  savings  while  a  learning  goal  may  be  to  master  efficient  flying  in  all 
aspects  of  being  a  pilot  (not  just  having  a  fuel  savings  focus  but  to  become  a  better  overall  pilot). 
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The  problem  with  learning  goals  is  that  they  can  potentially  conflict  with  goal  specificity  if  they 
are  written  too  vague.  Careful  consideration  of  the  target  group  will  provide  a  clear  picture  of 
what  type  of  goal  is  appropriate. 

Pitfall  two  focuses  on  group  dynamics  and  the  potential  for  conflict  among  members. 

The  authors  state  “when  two  or  more  people  believe  that  their  goals  are  competitively  rather  than 
cooperatively  related,  they  are  likely  to  be  tempted  to  pursue  their  own  goals  single-mindedly” 
(Latham  and  Locke,  2006:334).  Withholding  infonnation  is  a  serious  concern  if  the  goal  is 
viewed  as  purely  competitive.  This  seems  contrary  to  the  logging  example  in  which  competition 
was  lauded  as  a  facilitator  to  performance.  The  key  is  to  foster  competition  while  simultaneously 
promoting  collaboration.  One  way  this  is  accomplished  is  to  make  sure  the  goals  do  not  provide 
an  incentive  to  withhold  collaboration.  As  an  example,  if  AC’s  were  promoted  based  on  relative 
standing  with  fuel  efficiency,  they  would  not  want  to  help  their  competition. 

Pitfalls  three  and  four  are  interrelated  and  have  to  do  with  personal  threats  and 
punishment.  If  a  person  views  a  goal  as  a  threat,  performance  goes  down.  The  same  result  is 
seen  if  a  person  is  punished  for  not  attaining  a  goal.  Framing  the  goal  in  a  positive  way  and 
avoiding  punishment  are  essential  to  increase  performance  and  keep  individuals  from 
downgrading  specific,  difficult  goals  to  easier  goals. 

Pitfall  five  discusses  goal  attainment  and  long-term  reliance  on  successful  strategies,  even 
if  those  strategies  need  to  be  evolved.  This  is  not  an  issue  in  the  short-term  follow-on  research 
but  must  be  considered  in  the  future  if  any  policy  changes  occur  due  to  the  efficiency  research. 

Pitfall  seven  occurs  when  an  organization’s  identity  is  tied  to  a  goal,  causing  overreach 
and  over  commitment.  This  occurs  when  a  leader  or  group  of  leaders  will  do  anything  to  attain  a 
goal,  regardless  of  the  second  and  tertiary  effects.  In  the  flying  community,  this  is  dangerous  as 


21 


efficient  flying  is  not  the  number  one  priority.  Safety  of  flight  and  mission  requirements  should 
take  precedence  over  saving  fuel. 

Pitfall  eight  focuses  on  nongoal  dimensions  and  warns  against  ignoring  them.  The  risk  is 
that  an  individual  could  miss  other  important  factors  (outside  of  the  goal)  that  could  lead  to 
increased  performance  or  be  important  to  the  situation.  It  is  important  for  individuals  and  groups 
not  to  lose  focus  on  the  bigger  picture  of  goal  attainment. 

Pitfall  nine  states  that  stress  must  be  considered  when  setting  goals.  When  setting  goal 
difficulty,  stress  implications  should  be  factored  into  the  decision.  A  goal  should  be  difficult 
enough  to  maximize  performance  while  not  so  difficult  that  it  creates  undue  stress.  Dealing 
specifically  with  fuel  efficiency,  AC’s  are  under  a  significant  amount  of  stress  with  the  nature  of 
their  jobs  and  with  all  of  the  requirements  of  the  AF. 

The  final  pitfall  is  what  the  military  would  call  “goal  creep”.  If  an  individual  or  group 
attains  a  goal,  the  goal  is  then  raised  higher.  If  that  goal  is  attained,  it  is  raised  again.  At  some 
point  in  time,  the  goal  goes  from  difficult  to  impossible;  this  should  be  avoided.  This  pitfall  is 
closely  related  to  pitfalls  five  (goal  attainment)  and  nine  (stress).  While  a  single  goal  will  be  set 
for  the  follow-on  experiment,  the  danger  lies  in  leadership  not  reevaluating  goals  and  the 
potential  for  success  at  intennittent,  appropriate  points  in  time. 
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III.  Methodology 


Introduction 

The  purpose  of  this  chapter  is  to  provide  methodological  details  of  this  thesis  topic.  Two 
areas  are  covered  including  the  analyzed  data  and  the  method  of  analysis  for  the  data.  In  the  first 
section,  the  raw  fuel  tracker  data  provided  from  AMC/A3F  (Fuel  Efficiency  Office)  is  described. 
The  second  section  details  the  methods  employed  to  compare  metrics  as  well  as  the  methods 
used  to  determine  a  fuel  efficiency  goal  for  AC’s. 

Data  to  be  Analyzed 

The  field  data  are  found  in  a  continuously  updated  fuel  tracker  report  (Microsoft  Excel" ) 
that  is  submitted  upon  sortie  completion  to  AMC/A3F.  The  majority  of  the  data  collected  in  the 
fuel  tracker  are  from  a  fuel  tracker  worksheet  entered  by  the  aircraft  commander.  A  C-17 
worksheet  example  is  found  in  Appendix  A.  The  data  is  obtained  from  AMC/A3F.  While  5 1 
fields  are  present  in  the  Excel®  tracker  data,  M 1  Cargo  Adjusted  uses  8  fields  while  M2Regression  uses  5 
fields,  highlighted  in  Table  2  and  Table  3.  It  is  important  to  note  that  M2Regression  “cargo  actual” 
includes  1/2  of  all  onloaded  and  offloaded  in-air  fuel  received  as  well  as  the  weight  of  the  actual 
cargo  load.  The  name  column  shows  the  fields  as  seen  in  the  excel  report.  The  description 
provides  clarity  to  the  abbreviated  fields  and  acronyms. 
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Table  3:  M 1  cargo  Adjusted  Excel  Fields 


Field 

Name 

Description 

1 

D  ICAO 

Departure  Focation 

2 

A  ICAO 

Arrival  Focation 

3 

CARGO  PEN 

Planned  Cargo  in  Thousands  of  Pounds 

4 

CARGO  ACT 

Actual  Cargo  in  Thousands  of  Pounds 

5 

RAMP  FUEF  PEN 

Planned  Fuel  in  Thousands  of  Pounds 

6 

RAMP  FUEF  ACT 

Actual  Fuel  in  Thousands  of  Pounds 

7 

FAND  FUEF  PEN 

Planned  handing  Fuel  in  Thousands  of  Pounds 

8 

FAND  FUEF  ACT 

Actual  handing  Fuel  in  Thousands  of  Pounds 

Table  4:  M2Regression  Excel  Fields 


Field 

Name 

Description 

1 

CARGO  ACT 

Actual  Cargo  in  Thousands  of  Pounds 

2 

FEY  TIME  ACT 

Hours  of  Actual  Flight 

3 

RAMP  FUEF  PEN 

Planned  Fuel  in  Thousands  of  Pounds 

4 

RAMP  FUEF  ACT 

Actual  Fuel  in  Thousands  of  Pounds 

5 

FAND  FUEF  PEN 

Planned  handing  Fuel  in  Thousands  of  Pounds 

Method  of  Analysis 

Various  analyses  are  performed  in  order  to  assess  the  metrics  and  create  an  efficiency 
goal.  First,  MlCargo  Adjusted  formulation  is  described  as  well  as  the  factors  that  take  the  formulated 
metric  and  allow  it  to  be  used  as  an  efficiency  score  for  AC’s.  Second,  M2RegresSion  is  reviewed 
on  the  basis  of  regression  analysis.  Third,  Ml  cargo  Adjusted  and  M2Regressi0n  will  be  compared  using 
the  criteria  in  “the  Metric  Handbook”  (Nowak,  1992)  and  scoring  those  criteria  with  two  separate 
decision-based  matrices  found  in  “The  Quality  Toolbox”  (Tague,  2005).  Once  the  optimal 
metric  is  chosen,  GST  criteria  are  applied  to  the  metric  to  comprise  an  effective  efficiency  goal. 
Finally,  VBA  code  is  utilized  to  provide  feedback  to  AC’s  via  Excel®  in  the  follow  on  research. 
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First,  the  development  process  that  more  accurately  accounts  for  cargo  payload  in 
Ml  cargo  Adjusted  is  described.  Accounting  for  cargo  is  vital  because  if  payload  is  not  factored  into 
the  fuel  efficiency  scoring  metric,  AC’s  could  purposely  try  to  carry  less  cargo  to  attain  a  better 
efficiency  score.  Lack  of  cargo  accountability  makes  current  data  less  accurate  from  a  metric 
perspective.  Ml  cargo  Adjusted  fonnulations  take  payload  out  as  a  factor  in  AC  efficiency  score 
evaluation. 

ft  is  important  to  understand  the  methodology  that  derives  Ml  cargo  Adjusted  into  a  useable 
format  before  analyzing  its  efficacy  in  Chapter  IV.  Ml  cargo  Adjusted  consists  of  four  major 
components  illustrated  in  Equation  2: 

a  =  yact-yPl ~s  (2) 

Where: 

CL  =  Fuel  Efficiency  Goal  Metric  (AC  Efficiency  Score) 

yact  =  Fuel  Consumed  Actual: 

=  Actual  Ramp  Fuel  -  Actual  Landing  Fuel 

y pi  =  Fuel  Consumed  Planned 

=  Planned  Ramp  Fuel  -  Planned  Landing  Fuel 

S  =  Change  in  Fuel  Consumed  due  to  Actual  Payload 

As  seen  in  Table  3,  the  actual  ramp/actual  landing  and  planned  ramp/planned  landing  fuel  are 
provided  in  the  tracker  data.  Next,  the  change  in  fuel  consumed  due  to  actual  payload  is 
computed.  Work  from  Reiman  (2014)  aids  in  the  formulation  of  this  component  of  the  metric, 
shown  in  Appendix  B  (equations  22  &  23).  Reiman’s  research  uses  regression  analysis  to 
determine  aircraft  fuel  consumption  given  distance  and  payload.  In  Excel®,  thirty-two 
additional  fields  are  attached  to  the  end  of  the  Figure  4  fuel  tracker  fields  in  order  to  facilitate  the 
additional  formulas  that  incorporate  the  efficiency  metric. 


25 


Next,  M2  Regression  assumptions  are  tested  based  on  the  AMC/A3F  regression  coefficients 
to  check  for  validity.  Fuel  tracker  data  paired  down  to  C-17  SAAM  missions  from  active  duty 
wings  are  used  to  test  the  regression  perfonnance  (Sep  1 1-Augl4).  Regression  assumptions  of 
normality,  constant  variance,  and  independence  will  be  tested  for  validity. 

Finally,  a  comparison  of  metrics  is  perfonned  in  relation  to  Active  Duty  Wings  to 
determine  how  the  metrics  perform  with  actual  data.  Several  statistical  comparisons  are 
performed  to  judge  the  overall  perfonnance  of  the  metrics  in  comparison  to  each  other. 
Descriptive  statistics  are  evaluated  including  standard  deviation,  variance,  and  range.  Mean 
efficiency  scores  in  relation  to  geographic  location  as  well  as  time-series  are  evaluated  to  look 
for  differences  between  the  metrics. 

Third,  GST  criteria  of  content  (difficulty/specificity)  and  intensity  (commitment)  are 
coupled  with  the  chosen  metric  to  establish  an  efficiency  goal.  Previous  GST  literature  provides 
examples  and  case  studies  to  aid  in  the  establishment  of  an  effective  goal.  Once  the  quantitative 
goal  is  set,  considerations  of  feedback  and  GST  pitfalls  are  also  be  considered. 

Finally,  once  the  selected  metric  is  calculated  for  the  AC’s,  a  VBA  code  will  be  required 
in  the  follow-on  research  to  filter  the  vast  amounts  of  data  into  simplified  reports  capable  of 
being  sent  out  to  the  AC’s  being  evaluated. 
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IV.  Results 


In  this  chapter,  the  evaluation  results  from  Ml  cargo  Adjusted  and  M2Regression  are  described. 
First,  outliers  are  addressed.  Second,  the  replicated  regression  analysis  of  M2Regression  is 
performed.  Third,  the  Metric  Handbook  model  is  discussed  in  relation  to  M 1  Cargo  Adjusted  and 
M2Regression.  Finally,  the  researcher  will  compare  the  metrics  and  their  perfonnance. 

Outliers 

To  facilitate  this  research  as  well  as  the  follow-on  research,  outliers  are  considered.  After 
reviewing  a  fuel  tracker  data  set  from  September  2013  to  August  2014,  it  is  determined  that 
sorties  outside  of  +-  3  standard  deviations  are  largely  the  result  of  improperly  input  fuel  tracker 
fields  or  the  result  of  other  mitigating  circumstances  that  disproportionately  impact  the  fuel 
efficiency  score.  For  instance,  an  AC  may  have  written  a  note  that  the  flight  was  diverted  but  did 
not  properly  code  the  flight  as  diverted.  Another  example  of  an  outlier  is  if  a  flight  received  in¬ 
air  refueling  making  it  an  extremely  efficient  flight.  Based  on  this  review,  histograms  are 
generated  for  each  metric,  as  shown  in  Figure  3  and  Figure  4.  Overall  151  data  points  are 
removed  from  the  total  of  n=5727  to  get  a  total  reduction  of  2.7%  of  the  data  points. 
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Figure  3:  M 1  Cargo  Adjusted  Histogram  Sep  13  to  Aug  14 


Histogram  M2  Efficiency  Score  Sep  13  -  Aug  14 


Bin 


Figure  4:  M2RegresSi0n  Histogram  Sep  13  to  Aug  14 
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Due  to  time  constraints,  it  was  determined  that  rather  than  investigate  every  outlier,  an  across- 
the-board  approach  based  on  standard  deviations  would  appropriately  mine  and  provide  validity 
to  the  data. 

Regression 

The  original  M2Regression  regression,  performed  by  AMC/A3F,  utilized  C-17  sorties  for 
FY10  (10ctlO-30Sepl  1).  This  data  was  not  available  so  the  researcher  instead  used  data  for 
FY1 1  (1  Oct  1  l-30Sepl2)  to  determine  if  the  regression  analysis  proved  viable  to  detennine  fuel 
efficiency.  The  original  work  from  AMC  did  not  specify  if  the  data  was  paired  down  in  any  way 
such  as  location,  outliers,  etc.  so  the  assumption  is  made  that  the  analysis  was  perfonned  on  all 
Wings  including  Guard  and  Reserve  and  that  no  outliers  were  removed  from  the  data  set  prior  to 
the  regression  analysis. 

The  replicated  regression  analysis  was  performed  in  JMP®  statistical  software.  The 
initial  results  showed  a  similar  R-Square  to  the  AMC  results  of  .97.  To  further  validate  the 
results,  multicollinearity  and  regression  assumptions  needed  to  be  addressed  including  constant 
variance,  normality,  and  independence  to  ensure  the  entire  model  is  valid. 

First,  a  multicollinearity  check  is  performed  by  examining  a  multivariate  plot  in  JMP® 
and  the  variance  inflation  factors  (VIF)  within  the  output.  Several  trends  are  found  in  the  plot 
suggesting  some  multicollinearity  (see  Appendix  F).  Squared  and  interaction  terms  are  expected 
to  show  a  trend  as  are  any  terms  associated  with  the  y  variable  (actual  fuel  consumed).  An  issue 
of  multicollinearity  would  occur  if  actual  fly  time  and  actual  cargo  showed  a  trend  but  they  do 
not.  VIF  scores  provide  a  further  check  for  multicollinearity  as  they  “descrive  how  much 
multicollinearity  (correlation  between  predictors)  exists  in  a  regression  analysis  (Minitab,  2015). 
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A  VIF  score  above  five  causes  concern  while  a  VIF  score  above  10  strongly  suggests 
multicollinearity.  Figure  7  shows  the  JMP®  output  and  corresponding  VIF  scores. 


A  (2  Response  Actual  Fuel  Consumed 


a  Summary  of  Fit 

RSquare 

0.987902 

RSquare  Adj 

0.987891 

Root  Mean  Square  Error 

5.954382 

Mean  of  Response 

72.50552 

Observations  (or  Sum  Wgts) 

5575 

Analysis  of  Variance 

Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

5 

16122610 

3224522 

90947.76 

Error 

5569 

197447 

35 

Prob  >  F 

C.  Total 

5574 

16320057 

<0001* 

41  Lack  Of  Fit 


Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Lack  Of  Fit 

4012 

168714.52 

42.0525 

2.2788 

Pure  Error 

Total  Error 

1557 

5569 

28732.49 

197447.01 

18.4537 

Prob  >  F 

<0001* 

Max  RSq 

0.9982 

4)  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

VIF 

Intercept 

2.8143192 

0.212053 

13.27 

<0001* 

Cargo(Cargo  Act+(.5*Offload/Onload)  klbs 

0.0275158 

0.008145 

3.38 

0.0007* 

10.286082 

Cargo  ActA2 

-0.000369 

9.07e-5 

-4.07 

<0001* 

9.5341464 

Fly  Time  Act 

14.479448 

0.102467 

141.31 

<0001* 

15.48852 

Fly  Time  ActA2 

0.1260845 

0.009843 

12.81 

<0001* 

14.809766 

Cargo*FlyTime 

0.0381797 

0.000955 

39.98 

<0001* 

6.0076535 

Figure  5:  Summary  Statistics  Output  M2Regressi0n  Replication 


Next,  the  assumption  of  constant  variance  is  essential  to  validate  the  regression.  A  check 


of  the  predicted  and  residual  values  will  show  if  there  are  any  variance  problems.  As  seen  in 


Figure  10,  the  data  exhibits  some  curvature,  violating  the  constant  variance  assumption. 
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Figure  6:  Predicted/Residual  Constant  Variance  Check  M2Regression  Replication 


The  next  assumption  of  normality  can  be  checked  in  JMP®  using  a  nonnal  quantile  plot. 
If  the  regression  exhibits  normality,  the  data  would  show  a  linear  trend.  If  there  is  any  curvature 
in  the  graph,  nonnality  is  violated.  As  seen  in  Figure  11,  the  normality  assumption  is  violated  as 
the  normal  quantile  plot  shows  definitive  curvature  in  the  trendline. 


Figure  7:  M2Regression  Replication  Normality  Test 
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The  final  assumption  that  needs  to  be  addressed  is  independence.  A  time  series  vs. 
residual  scatterplot  performs  this  check.  Figure  12  is  a  one-week  snapshot  of  the  data;  the  entire 
data  set  is  too  complex  to  look  for  trends  in  a  scatterplot.  The  figure  shows  that  the  data  appears 
to  pass  the  test  for  independence  as  it  does  not  show  specific  trends. 


Time  Series  vs.  Residuals 


Date 


Figure  8:  Independence  Check  M2  Regression  Replication 
To  further  examine  this  finding,  a  Durbin  Watson  test  was  perfonned.  JMP®  statistical  software 

calculated  a  d-statistic  of  1.839.  Based  on  an  alpha  of  0.05,  critical  values  of  dL, alpha  =1-72  and 

du,aipha=l  .82,  and  a  4-d  value  of  2. 1 7,  autocorrelation  does  exist  within  the  data.  Therefore,  the 

test  for  independence  fails. 

Metric  Handbook 

With  the  statistical  analysis  of  M2  Regression  completed,  an  analysis  of  “The  Metric 
Handbook”  model  (Nowak,  1992:2-1)  in  relation  to  each  metric  will  provide  an  overall  picture 
on  how  the  metrics  perfonn.  For  the  purposes  of  this  section,  A1-A6  will  signify  attribute  1- 
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attribute  6  in  the  model.  It  is  important  to  note  that  this  analysis  is  being  performed  through  the 
lens  of  using  this  metric  to  influence  AC  behavior  in  future  motivational  experiments. 


M 1  Cargo  Adjusted  • 


Al:  It  is  accepted  as  meaningful  to  the  AC’s? 

Undecided,  Ml  cargo  Adjusted  accounts  for  factors  outside  of  the  AC’s’  control. 
Payload  is  accounted  for  through  comprehensive  calculations.  Other  ACFP 
factors  included  in  Ml  cargo  Adjusted  are  tail  fuel  degrade,  air  traffic  control  hold 
down,  thunderstorms  and  turbulence  fuels,  planned  air  refueling,  planned  flight 
route,  planned  altitude,  forecasted  temperatures  and  winds 

A2:  Is  It  Unambiguously  defined? 

Yes,  the  metric  entails  the  difference  between  planned  fuel  consumed  adjusted  for 
actual  payload  and  actual  fuel  consumed. 

A3:  It  tells  how  well  organizational  goals  and  objectives  are  being  met  through 
processes  and  tasks? 

Yes,  once  efficiency  goals  are  set,  AC’s  will  have  a  clear  measure  to  meet  the  set 
goal.  If  AC’s  are  not  meeting  organizational  goals,  it  will  be  clearly  evident  in 
the  M 1  cargo  Adjusted  calculations  and  graphical  trends. 

A4:  It  is  simple,  understandable,  logical,  and  repeatable? 

Undecided,  once  the  calculations  are  embedded  into  excel  spreadsheets  or  VBA 
code,  it  is  easy  to  follow  and  utilize.  The  formulas  are  lengthy  and  prone  to 
mistakes  if  attention  to  detail  is  not  prevalent.  Embedding  the  calculations  into 
VBA  or  other  coding  program  could  reduce  a  lot  of  errors  and  simplify  Ml.  Ml 
is  derived  from  ACFP  which  is  well  understood  planning  software  for  the  flying 
community. 

A5:  Its  data  is  timely  and  economical  to  collect. 

Yes,  simple  spreadsheet  calculations  coupled  with  fuel  tracker  data  can  be 
perfonned  if  the  formulas  are  already  coded. 

A6:  It  drives  the  appropriate  action. 

Undecided,  a  subsequent  discussion  is  provided  at  the  end  of  this  analysis. 
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M2  Regression  • 

Al:  Is  it  accepted  as  meaningful  to  the  AC’s? 

Undecided,  M2  Regression  fails  to  take  into  account  ACFP  planning  factors  such  as 
tail  degrade,  winds,  temperature,  planned  air  refueling,  planned  routing,  planned 
altitude,  turbulence,  thunders tonns,  icing,  and  air  traffic  control.  The  simplistic 
approach  of  using  time  and  cargo  to  predict  fuel  bum  might  not  have  adequate 
buy-in  from  the  flying  community. 

A2:  Is  It  Unambiguously  defined? 

Yes,  M2  Regression  has  a  clear  definition  and  understandable  construct.  The 
variables  used  are  clear  and  known  to  AC’s.  From  a  definition  perspective,  the 
metric  is  understandable. 

A3:  It  tells  how  well  organizational  goals  and  objectives  are  being  met  through 
processes  and  tasks  it  shows  a  trend. 

No,  it  contrasts  perfonnance  to  a  201 1  baseline  based  on  time  of  flight  and  gross 
weight.  There  are  many  other  planning  factors  outside  of  the  AC’s  control  that 
are  assessed  through  ACFP  that  are  not  considered  by  the  regression. 

A4:  It  is  simple,  understandable,  logical,  and  repeatable? 

Undecided,  M2Regression  meets  these  criteria  if  the  user  has  an  understanding  of 
regression  analysis  and  its  application.  AC’s  have  an  understanding  of  the  fuel 
tracker  and  how  it  works  but  if  they  cannot  make  the  transition  from  tracker  input 
to  regression  output,  simplicity  and  ease  of  use  will  be  degraded. 

A5:  Its  data  is  timely  and  economical  to  collect. 

Yes,  AC’s  are  already  required  to  input  fuel  tracker  data  and  a  system  is  in  place 
for  AMC  to  collect  this  data.  Any  statistical  software  package  or  Microsoft 
Excel®  can  quickly  transform  the  fuel  tracker  data  into  a  usable  form  of 
M2  Regression.  It  is  simple  and  user-friendly  to  perform  regression  analysis  with  a 
basic  understanding  of  the  steps  and  process  surrounding  the  technique. 

A6:  It  drives  the  appropriate  action. 

Undecided,  a  subsequent  discussion  is  provided  at  the  end  of  this  analysis. 


34 


Attribute  6  is  the  most  important  aspect  of  each  of  the  metrics.  The  ability  of  each  metric 
to  drive  action  and  behavior  will  have  the  largest  impact  in  fuel  efficient  flying.  For  this  reason, 
flight  factors  both  inside  and  outside  of  an  AC’s  control  will  be  addressed  in  relation  to  each 
metric.  An  examination  of  the  impact  of  temperature,  winds,  routing,  speed,  and  altitude  will 
provide  a  distinct  comparison  between  the  metrics  and  the  behavior  they  induce. 

ACFP  uses  forecasted  temperature  which  influences  Ml  cargo  Adjusted-  This  allows  AC’s  a 
moderate  amount  of  control  to  fly  an  alternate  route  that  may  have  lower  average  temperatures, 
increasing  fuel  efficiency.  Conversely,  M2  Regression  does  not  account  for  temperature  which 
minimizes  AC  control.  For  example,  an  AC  flying  out  of  northern  (cold  weather)  city-pairs  will 
have  a  higher  efficiency  than  an  AC  flying  in  southern  (wann  weather)  city-pairs.  Under 
M2  Regression,  an  AC  could  look  for  the  more  favorable  temperatures,  similar  to  Ml  cargo  Adjusted,  but 
it  will  still  be  very  disproportionately  negative  in  wanner  climates. 

Next,  forecasted  winds  are  included  in  ACFP.  Under  Ml  cargo  Adjusted,  AC’s  may  find 
significant  improvement  based  on  finding  lower  headwinds  than  the  planned  route  is 
recommending.  M2Regression  does  not  account  for  winds  in  the  regression;  instead,  the  metric 
factors  flight  time  which  negatively  impacts  the  AC  in  the  case  of  high,  inefficient,  headwinds. 
The  AC  benefits  heavily  from  favorable  headwinds  due  to  decreasing  flight  time  (regression 
variable)  but  is  more  heavily  penalized  from  going  into  the  headwinds.  Under  M2,  the  benefit 
from  finding  a  lower  headwind  on  improved  fuel  consumption  is  offset  by  the  decreased  flight 
time’s  reduction  of  the  planned  fuel  consumed.  M2  Regression  will  favor  crews  going  east  and 
penalize  crews  going  west  due  to  winds. 
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Routing  considerations  also  favor  M 1  cargo  Adjusted  because  the  AC  may  ask  for  a  more 
favorable  direct  route  based  on  experience.  M2  Regression  does  not  contain  routing  factors  as  a 
regression  variable  so  an  AC  may  choose  whatever  route  can  be  accomplished  in  the  shortest 
amount  of  time  rather  than  the  most  fuel  efficient  route.  Getting  a  direct  route  with  M2  Regression 
will  reduce  the  actual  fuel  consumed,  but  will  also  reduce  the  planned  fuel  consumed  by 
decreasing  the  time  component  of  the  regression. 

Planned  altitude  is  also  included  in  ACFP  which  could  benefit  AC’s  under  Ml  cargo  Adjusted- 
A  higher  altitude  is  often  more  efficient  so  the  AC  may  choose  a  higher  altitude  to  fly  at  to  gain 
fuel  efficiency.  A  step  climb  is  another  option  for  an  AC  in  which  an  AC  climbs  to  maximum 
altitude  based  on  weight  and  then  gradually  climbs  as  weight  is  lost  due  to  fuel  bum.  M2  Regression 
does  not  factor  in  altitude  so  step  climb  tactic  would  not  necessarily  benefit  them.  Climbing  to  a 
higher  altitude  will  often  decrease  the  time  enroute  by  achieving  a  greater  true  airspeed.  The 
decrease  in  time  will  decrease  the  planned  fuel  consumed  often  negating  the  fuel  efficiency 
benefit  from  the  higher  altitude. 

Overall,  there  is  a  sharp  contrast  between  Ml  cargo  Adjusted  and  M2RegreSsion  in  relation  to 
whether  the  metrics  drive  the  appropriate  behavior.  Ml  cargo  Adjusted  incentivizes  efficient  flying 
behaviors  by  including  planning  factors  such  as  temperature,  winds,  routes,  and  altitude. 
Essentially,  the  factors  that  are  outside  of  an  AC’s  control  that  are  due  to  the  uniqueness  of  the 
sortie  are  removed  for  Ml  cargo  Adjusted  but  not  for  M2  Regression-  This  allots  AC’s  the  opportunity  to 
utilize  their  knowledge,  skills,  and  abilities  (KSA’s)  to  try  and  garner  efficiency  above  and 
beyond  ACFP  recommendations  without  circumventing  efficient  maneuvers  to  “play  to  a 
metric.”  Ml  cargo  Adjusted  efficiency  is  not  dependent  on  payload.  It  also  increases  if  actual  ramp 
fuel  is  reduced  (lighter  aircraft)  or  actual  landing  fuel  is  increased  (fuel  conservation). 
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In  order  to  quantify  A1-A6,  two  decision  matrices  were  taken  from  “The  Quality 
Toolbox”  written  by  Nancy  Tague  (2005).  Weighted  (Table  2)  and  rank-order  matrices  (Table 
3)  will  provide  a  score  to  each  metric  based  on  the  six  criteria  found  in  “The  Metric  Handbook.” 
Based  on  the  scores  of  22  and  15  (Method  1)  compared  to  33  and  30  (Method  2),  Ml  cargo  Adjusted 
is  shown  to  be  the  more  optimal  choice  when  quantitatively  analyzed  in  regards  to  the  metric 
criteria. 


Table  5:  Weighted  Matrix  (Fuel  Efficiency  Metrics) 


Method  1  (Weighted)  Decision  Matrix:  Fuel  Efficiency  Metric 

Criteria 

Problems  j 

Meaningful/ 

Unambigulously 

Defined 

3 

Organizational 

Goals 

Being  Met 

2 

Simple,  Understandable,  Logical, 
Repeatable 

2 

Economical 

Collection/Time 

iy 

l 

Drives  Appropriate 
Action 

3 

Total 

s 

Metric  1 

Medium 

High 

Medium 

High 

High 

28 

3X2=6 

2X3=6 

2X2=4 

1X3=3 

3X3=9 

Metric  2 

Low 

Low 

Medium 

High 

Low 

15 

3X1=3 

2X1=2 

2X2=4 

1X3=3 

3X1=3 

Table  6:  Rank  Order  Matrix  (Fuel  Efficiency  Metrics) 


Method  2  (Rank  Order)  Decision  Matrix:  Fuel  Efficiency  Metric 

Criteria 

Problems  i 

Meaningful/ 

Unambigulously 

Defined 

3 

Organizational 

Goals 

Being  Met 

2 

Simple,  Understandable,  Logical, 
Repeatable 

2 

Economical 

Collection/Time 

iy 

l 

Drives  Appropriate 
Action 

3 

Total 

s 

Metric  1 

2X3=6 

3X2=6 

1X2=2 

4X1=4 

5X3=15 

33 

Metric  2 

3X3=9 

1X2=2 

4X2=8 

5X1=5 

2X3=6 

30 

Metric  Comparison 

The  next  analysis  regarding  the  metrics  will  compare  the  perfonnance  of  the  metrics 
using  actual  fuel  tracker  data  from  2011-2014.  First,  the  summary  statistics  will  be  examined  to 
demonstrate  the  differences  in  variability  as  well  as  the  range  of  data.  Second,  the  metrics  will 
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be  compared  at  the  Wing  level  to  detennine  if  Wings  perform  differently  from  each  other.  Third, 
questions  of  tail  impact  and  location  will  be  examined  from  an  individual  AC  perspective  to  see 
if  aircraft  or  location  causes  differences  in  AC  fuel  efficiency.  Finally,  an  examination  of  AC 
perfonnance  based  on  percentiles  will  allow  an  overview  of  how  efficiency  scores  from  the  most 
and  least  efficient  AC’s  compare.  This  will  be  vital  in  setting  the  final  efficiency  goal. 

An  initial  Excel®  summary  statistics  examination  of  the  metrics  showed  significant 
differences  in  variance  and  mean  values.  Shown  in  Table  4,  M2Regression  exhibits  a  higher 
variance  and  mean  than  Ml  cargo  Adjusted-  Ml  cargo  Adjusted  scores  are  more  efficient  on  average  at 
0.84  klbs  while  M2  suggests  more  inefficient  flying  with  a  mean  score  of  over  1.4klbs. 

Table  7:  Summary  Statistics  Ml  and  M2  Sep  13  -  Aug  14  (SAAM  missions,  Outliers  Removed) 


Ml 

M2 

Mean 

-0.84 

Mean 

1.42 

Standard  Enor 

0.06 

Standard  Error 

0.08 

Median 

-0.66 

Median 

1.44 

Standard  Deviation 

4.44 

Standard  Deviation 

6.16 

Sample  Variance 

19.72 

Sample  Variance 

37.97 

Range 

48.65 

Range 

61.48 

Minimum 

-25.99 

Minimum 

-29.79 

Maximum 

22.66 

Maximum 

31.69 

Sum 

-4671.87 

Sum 

7912.34 

Count 

5576.00 

Count 

5576.00 

Confidence  Level(95.0%) 

0.12 

Confidence  Level(95.0%) 

0.16 

The  next  analysis  will  focus  on  the  differences  in  metric  performance  in  relation  to  Wings 
and  individual  AC’s.  Several  areas  will  be  examined  including  the  mean  efficiency  scores 
between  installations,  time-series  mean  efficiency  scores,  and  how  individual  AC’s  compare 
from  a  percentile  perspective.  Taken  together,  these  analyses  will  provide  better  insight  into 
metric  perfonnance  and  potential  problems  in  the  metrics  from  a  trend  perspective. 
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Beginning  with  an  examination  of  the  C-17,  active  duty  Wings  in  relation  to  Ml  cargo 
Adjusted  and  M2  Regression?  the  researcher  will  show  whether  any  significant  differences  (or 
similarities)  exist  among  the  Wings.  Figure  13  identifies  the  mean  efficiency  score  (SAAM 
missions)  for  each  Wing  over  a  12  month  period  (Sepl3-Augl4).  M2  showed  significantly 
higher  averages  while  all  mean  Ml  cargo  Adjusted  values  were  +-  1.35Klb.  Hold-down  fuel  could 
contribute  to  some  of  the  results;  according  to  a  fuel  policy  letter  from  November  2014  (Gillson), 
the  two  most  efficient  M 1  cargo  Adjusted  scores  (Dover,  McGuire)  both  receive  4,500  lbs  of  hold¬ 
down  fuel  due  to  busy  ATC  conditions.  The  added  fuel  could  be  contributing  to  the  increased 
efficiency  by  allotting  the  AC’s  more  fuel  than  they  bum,  beating  the  planned  landing  fuel  and 
increasing  efficiency.  Charleston  could  also  be  very  efficient  due  to  this  factor;  sorties  leaving 
Charleston  often  transit  Dover  or  McGuire  and  are  subject  to  the  hold-down  fuel  addition. 

M2  Regression  performs  in  a  similar  fashion  with  Dover,  McGuire,  and  Charleston  being  the  most 
efficient.  Appendix  F  contains  the  policy  letter  details. 
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Sep  13-Augl4  Mean  Efficiency  Score  Metric  1/Metric  2 

3.50  - 


-1.50  - 

Base 


Figure  9:  M 1  Cargo  Adjusted  &  M2  Regression  Mean  Efficiency  Scores 
The  most  efficient  (McGuire)  and  least  efficient  (Elmendorf)  Wings  based  on  the  mean 

value  over  a  12  month  period  amounted  to  a  range  of  (-1.35,  0.57)  for  Ml  cargo  Adjusted-  The  most 
efficient  for  M2  is  McGuire  and  the  least  efficient  is  McChord  with  a  range  of  (0.16,  3.13).  To 
more  accurately  determine  if  these  ranges  are  statistically  significant,  a  comparison  of  means  was 
perfonned  in  Excel "  .  Table  5  output  statistics  show  statistically  significant  p-values  based  on  a 
0.05  alpha  value.  The  p-values  show  that  the  mean  values  are  statistically  different.  This  is 
important  because  different  mean  values  may  make  it  difficult  for  a  single  goal  to  be  set  for  all 
Wings. 
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Table  8:  M1/M2  Most/Least  Efficient  Bases  Mean  Comparison 


MI :  Most/Least  Efficient  Wing  Comparison 

M2 :  Most/Least  Efficient  Wing  Comparison 

z-Test:  Two  Sample  for  Means 

z-Test:  Two  Sample  for  Means 

Variable  1 

Variable  2 

Variable  1 

Variable  2 

Mean 

-1.35 

0.57 

Mean 

0.16 

3.13 

Known  Variance 

20.61 

15.77 

Known  Variance 

32.73 

45.67 

Observations 

644.00 

266.00 

Observations 

644.00 

970.00 

Hypothesized  Mean  Difference 

0.00 

Hypothesized  Mean  Difference 

0.00 

z 

-6.35 

z 

-9.49 

P(Z<=z)  one-tail 

0.00 

P(Z<=z)  one-tail 

0.00 

z  Critical  one- tail 

1.64 

z  Critical  one-tail 

1.64 

P(Z<=z)  two-tail 

0.00 

P(Z<=z)  two-tail 

0.00 

z  Critical  two-tail 

1.96 

z  Critical  two-tail 

1.96 

The  next  analysis  examines  Ml  cargo  Adjusted  and  M2Regression  performance  over  a  12  month 
time-series  from  Sep  13  to  Aug  14.  Originally,  the  data  set  contained  3  years  worth  of  data. 
Large  fluctuations  occurred  in  the  data  from  2011  to  2014  (see  3  year  chart  output  in  Appendix 
D)  suggesting  outside  factors  skewed  the  efficiency  trends.  While  the  exact  cause  of  the  changes 
are  difficult  to  pinpoint,  Mr.  Joe  Jackson,  the  Flight  Manager  from  the  618  AOC/XOCM,  points 
to  Flight  Crew  Information  File  (FCIF)  changes  as  a  potential  reason  for  fluctuations.  These 
changes  include  descent  fuel  calculation  alterations,  additional  fuel  authorizations,  and 
modifications  to  the  fuel  tracker  itself. 

This  highlights  the  importance  of  examining  external  factors  when  looking  at  fuel 
efficiency  metrics.  Policy  changes  can  significantly  impact  metrics,  trends,  and  how  overall 
performance  is  viewed.  Appendix  E  contains  FCIF  changes  that  may  have  influenced  metric 
perfonnance. 

The  data  becomes  more  consistent  when  the  last  12  months  of  data  is  viewed  together 
rather  than  the  entire  three  years.  Figure  14  suggests  much  greater  fuel  savings  from  Ml  cargo 
Adjusted  while  M2  Regression  is  significantly  less.  Appendix  D  contains  the  three  year  data  set  with 
outliers  removed. 
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Figure  10:  Time  Series  Metric  Comparison 

Finally,  individual  AC  performance  will  be  examined.  A  statistical  means  comparison 


was  performed  on  individual  AC’s  that  flew  multiple  tail  numbers  at  the  same  location  to  see  if 
differences  existed  in  aircraft  performance.  10  AC’s  were  found  that  flew  more  than  10  sorties 
on  two  different  aircraft.  The  means  comparison  proved  to  be  inconclusive  with  approximately 
half  of  the  AC’s  having  statistically  similar  means  while  the  other  half  had  statistically  different 
means,  seen  in  Table  6.  A  location  analysis  (same  AC  at  different  locations)  was  unable  to  be 
performed  due  to  sample  size  constraints. 
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Table  9:  Aircraft  Tail  Analysis 


Same  AC,  Same  Squadron,  2  Different  Tails  Mean  Comparison  p-values 

Pilot  # 

Ml 

M2 

#  Sorties/Tail 

1 

0.69 

0.43 

11/10 

2 

0.57 

0.94 

10/11 

3 

0.21 

0.17 

12/10 

4 

0.20 

0.14 

16/14 

5 

0.58 

0.01 

22/13 

6 

0.02 

0.0015 

11/21 

7 

0.03 

0.28 

10/10 

8 

0.02 

0.00 

10/24 

9 

0.0013 

0.000062 

10/24 

10 

0.21 

0.33 

23/12 

The  final  analysis  will  focus  on  percentile  rankings  and  fuel  efficiency.  This  analysis 
will  provide  significant  information  that  will  aid  in  setting  an  appropriate  goal  for  AF  AC’s. 

M2 Regressi0n  will  not  be  analyzed  in  this  section  because  this  information  is  primarily  aimed  at  the 
follow-on  research  and  percentile  calculations  can  be  applied  to  any  metric. 

Initially,  all  sorties  were  evaluated  without  regards  to  specific  AC’s.  AF  percentiles  were 
calculated  for  SAAM  missions  with  outliers  removed  from  Sep  13-  Aug  14.  The  results  in  Table 
7  show  the  breakdown  of  all  AF  fuel  efficiency  percentiles  over  a  total  of  5576  sorties. 
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Table  10:  AF  Percentiles  SAAM  Mission  Klbs/fuel 


AF  Percentiles 

0 

-25.99 

0.1 

-5.78 

0.2 

-3.73 

0.3 

-2.54 

0.4 

-1.61 

0.5 

-0.66 

0.6 

0.008 

0.7 

0.9 

0.8 

2.15 

0.9 

4.12 

1 

22.66 

Next,  percentiles  were  calculated  for  each  Wing.  This  was  done  so  that  the  Wings  could 
be  compared  to  each  other  and  against  overall  AF  performance.  The  most  interesting  finding  is 
that  the  bottom  50th  percentile  of  sorties  was  very  consistent  across  all  Wings.  The  top  50th 
percentile  differed  in  that  the  Wings  varied  more.  The  top  10%  of  all  sorties  varied  by  -3.44klbs 
between  the  most  and  least  efficient  Wings.  Additionally,  the  Wings  differed  from  each  other 
and  from  the  AF.  For  instance,  the  top  10%  of  flights  from  the  305th  Wing  at  McGuire  had  a 
score  of  -7.18klbs  (Table  8)  while  the  AF  percentile  was  -5.78klbs  (Table  7). 


Table  11:  MlCargo  Adjusted  Klbs  Percentiles  All  Wings 


Ml  Klhs/Fuel  Percentiles/Wing  (SAAM,  No  Outliers) 

Percentiles 

15WG 

305WG 

3  WG 

436  WG 

437  WG 

60  WG 

62  WG 

Min 

Max 

Difference 

10th 

-3.96 

-7.18 

-3.74 

-6.95 

-5.85 

-4.36 

-4.86 

-7.18 

-3.74 

-3.44 

20th 

-2.58 

-4.68 

-1.71 

-4.67 

-3.84 

-3.26 

-3.28 

-4.68 

-1.71 

-2.97 

30th 

-1.57 

-3.31 

-0.76 

-3.17 

-2.70 

-2.13 

-2.18 

-3.31 

-0.76 

-2.55 

40th 

-0.85 

-2.22 

-0.01 

-2.01 

-1.88 

-1.49 

-1.17 

-2.22 

-0.01 

-2.22 

50th 

-0.23 

-0.90 

0.60 

-0.90 

-0.95 

-0.62 

-0.43 

-0.95 

0.60 

-1.55 

60th 

0.33 

0.00 

1.57 

0.00 

-0.05 

0.01 

0.19 

-0.05 

1.57 

-1.61 

70th 

1.27 

0.59 

2.17 

0.81 

0.71 

0.73 

1.17 

0.59 

2.17 

-1.58 

80th 

2.54 

1.57 

3.20 

2.18 

2.02 

1.78 

2.23 

1.57 

3.20 

-1.63 

90th 

4.54 

3.53 

4.90 

4.01 

3.99 

3.63 

4.25 

3.53 

4.90 

-1.37 
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A  better  pairing  of  percentile  scores  and  efficiency  goals  may  be  attained  from  looking  at 
individual  AC’s  and  how  they  perfonn.  The  data  provided  24  AC’s  that  performed  over  30 
sorties  at  a  single  location.  The  percentiles  were  calculated  for  all  flights  involving  these  AC’s 
and  can  be  seen  on  the  right  side  of  Table  9.  Next,  the  mean  scores  were  calculated  for  each  AC 
and  placed  into  one  of  the  aforementioned  percentiles.  Utilizing  these  mean  scores  and 
percentiles  may  provide  better  insight  into  a  potential  efficiency  goal  because  these  are  the  AC’s 
with  the  most  flights  in  relation  to  their  mission.  If  a  percentile  score  is  picked  from  the  overall 
AF  population,  the  pilot  may  only  have  a  few  sorties  and  thus,  the  set  goal  could  be  misleading. 
Rather,  averaging  experienced  AC  sorties  and  using  that  score  as  a  benchmark  provides  a  more 
realistic  and  attainable  number  for  all  AC’s  to  achieve. 
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Table  12:  Percentile  Scores  AC's  with  >30  Sorties 


AC's  with  >30  Sorties  Mean  Fuel  Efficiency  Score 


AC# 

Mean 

AC  Mean  Percentile 

Percentiles  (All  SAAM  Sorties) 

1 

-4.21 

0.2 

0 

-23.46 

2 

-3.04 

0.3 

0.1 

-6.86 

3 

-2.55 

0.4 

0.2 

-4.10 

4 

-2.31 

0.4 

0.3 

-2.70 

5 

-2.27 

0.4 

0.4 

-1.84 

6 

-1.58 

0.5 

0.5 

-0.70 

7 

-1.45 

0.5 

0.6 

0.00 

8 

-1.43 

0.5 

0.7 

0.92 

9 

-1.19 

0.5 

0.8 

2.23 

10 

-1.11 

0.5 

0.9 

4.52 

11 

-1.06 

0.5 

1 

19.34 

12 

-0.90 

0.5 

13 

-0.87 

0.5 

14 

-0.82 

0.5 

15 

-0.67 

0.6 

16 

-0.65 

0.6 

17 

-0.42 

0.6 

18 

-0.36 

0.6 

19 

-0.32 

0.6 

20 

-0.12 

0.6 

21 

0.01 

0.7 

22 

0.15 

0.7 

23 

0.68 

0.7 

24 

0.87 

0.7 
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V.  Discussion 


With  the  results  of  this  research  in  place,  this  chapter  will  transition  to  a  discussion  of  the 
investigative  questions  described  in  Chapter  I  as  well  as  potential  future  research  topics. 
Specifically,  the  researcher  will  discuss  the  adequacy  of  Ml  Cargo  Adjusted  and  M2Regression  as  fuel 
efficiency  metrics,  how  the  metrics  perform  from  a  Wing  and  individual  perspective,  which 
metric  should  be  utilized  in  future  AF  operations,  and  what  the  specific  efficiency  goal  should  be 
for  AF  leaders  to  set. 

Investigative  Questions 

1.  From  a  metric  criteria  perspective,  is  Ml  cargo  Adjusted  or  M2Regression  a  more  adequate  measure 
of fuel  efficiency  for  AC’s? 

Ml  cargo  Adjusted  is  the  selected  metric  based  on  factors  outlined  in  “The  Metric  Handbook”. 
The  analysis  of  metric  criteria  using  the  decision  matrices  demonstrated  a  difference  in  how  the 
metrics  would  be  accepted,  and  instituted,  within  an  organization.  In  particular,  the  underlying 
formulations  of  Ml  Carg0  Adjusted  (taking  into  account  planning  factors)  clearly  define  and  provide 
meaning  to  the  metric.  An  understanding  of  the  relationship  between  ACFP  and  metric 
formulation  will  be  more  intuitive  to  AC’s. 

M2’s  is  useful  for  baseline  comparison  but  not  AC  fuel  efficiency.  The  replacement  of 
ACFP  planning  measurements  with  predicted  regression  fonnulations  doesn’t  allot  for  full  use  of 
a  system  designed  to  account  for  external  factors  outside  the  bounds  of  the  AC’s  control.  These 
factors  must  be  controlled  to  the  maximum  extent  possible  to  convince  AC’s  that  they  have  a 
locus  of  control  over  the  metric  and  what  it  aims  to  influence.  If  planning  factors  are  not  a 
consideration,  AC’s  will  likely  question  the  purpose  of  the  metric  and  why  ACFP  does  not  play  a 
larger  role  in  detennining  efficiency. 
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2.  From  a  metric  comparison  perspective,  how  do  Wings/Individual  AC’s  perform? 

The  summary  statistic  revealed  important  differences  between  the  metrics.  With  a  higher 
standard  deviation  and  variance  in  M2 Regression,  the  individual  sortie  scores  were  spread  further 
out  from  the  mean.  This  suggests  much  broader  efficiency  scores  across  the  AF.  Ml  cargo  Adjusted 
had  a  smaller  standard  deviation  by  approximately  1.7klbs  per  sortie  (outliers  removed)  showing 
a  lot  less  variability  in  how  AC’s  performed  from  an  efficiency  perspective.  This  is  very 
important  when  considering  how  to  use  GST  tenets  to  quantify  efficiency  goals  for  all  AC’s. 
Under  M2  Regression,  the  increased  variability  presents  more  of  a  challenge  when  determining  the 
appropriate  level  for  a  goal  to  be  set. 

The  comparison  at  the  Wing  level  as  well  as  the  time-series  showed  a  stark  difference 
between  the  metrics.  First,  it  showed  that  all  Wings  may  not  be  able  to  have  the  same  goal  set 
for  them.  Although  they  are  not  drastically  different,  there  are  statistical  differences.  Dependent 
upon  the  approved  Wings  for  the  experiment,  this  difference  should  be  evaluated  before 
detennining  a  final  goal.  Next,  the  time  series  illustrated  that  FCIF  changes  and  any  other 
factors  that  impact  the  manner  in  which  AC’s  fly  must  be  accounted  for  when  determining  an 
optimal  metric  and  efficiency  goals. 

3.  Which  metric  should  be  provided  to  AC’s  and  for  what  purpose? 

From  a  broader  perspective,  Ml  cargo  Adjusted  leverages  current  ACFP  system  capability  by 
utilizing  planning  factors  that  have  already  been  established.  The  additions  of  payload  and 
distance  adjustments  made  in  Ml  cargo  Adjusted  calculations  could  eventually  be  incorporated  into 
ACFP,  negating  the  need  for  separate  calculations.  This  would  provide  more  validity  to  the 
system,  strengthen  AC  commitment  to  the  efficiency  metric  and  goal,  and  more  accurately 
account  for  fuel  use  and  savings.  To  illustrate  the  savings  associated  with  M 1  cargo  Adjusted  fuel 
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efficiency,  a  simple  illustration  of  the  adjusted  metric  over  a  three  year  period  is  relevant.  Figure 
15  shows  a  continual  downward  trend  over  a  three  year  period.  The  costs  associated  with  this 
trend  are  highlighted  in  Table  10  which  shows  a  $19. 1M  cost  swing  from  2011  -  2014.  It  is 
important  to  understand  that  the  42083  sorties  captured  in  this  analysis  only  cover  Channel, 
SAAM,  and  Contingency  missions  at  Active  Duty  Wings  for  C-17’s.  The  magnitude  of  applying 
a  more  robust  metric  across  the  fleet  and  at  Reserve  Wings  has  the  potential  to  demonstrate 


significant  cost  savings  by  providing  a  more  accurate  efficiency  calculation. 
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Table  13:  Ml  cargo  Adjusted  Cost  Trend  Sep  1 1  -  Sep  14 


12  Month  Cost  Comparison  Ml  (Channel/SAAM/Contingency) 

1  Sep  11-31  Aug  12 

1  Sep  12-31  Aug  13 

1  Sep  13-31  Aug  14 

Klbs 

30,971 

3,510 

(4,530) 

lbs 

30,970,675 

3,510,151 

(4,530,229) 

Gallons 

4,622,489 

523,903 

(676,154) 

$/Gallon 

3.62 

3.62 

3.62 

Total  Cost 

$  16,733,409.23 

$  1,896,529.48 

$  (2,447,676.05) 

4.  Using  GST,  how  can  the  chosen  metric  be  presented  as  a  goal  for  AC’s?  What  specific 
efficiency  goal  can  be  set  for  AF  AC’s? 

As  discussed  in  Chapter  II,  content  (specificity,  difficulty)  and  intensity  (commitment) 
are  the  key  components  of  GST  that  will  increase  the  probability  of  higher  performance  and  goal 
attainment.  Commitment  will  be  heavily  reliant  on  the  design  and  execution  of  the  experiment. 
Specificity  and  difficulty  can  be  instituted  by  pairing  the  metric  and  the  percentile  analysis. 
Using  GST  principles  and  examples  (Latham  and  Locke  1991:213-214  and  Locke  et  ah, 
1989:272),  specificity  and  difficulty  can  be  paired,  illustrated  in  the  following  example: 

1 .  Easy:  Attain  a  mean  score  in  the  70th  percentile  by  achieving  a  5  sortie  moving 
average  of  lklbs. 

2.  Moderate:  Attain  a  mean  score  in  the  60th  percentile  by  achieving  a  5  sortie  moving 
average  of  Oklbs. 

3.  Difficult:  Attain  a  mean  score  in  the  30th  percentile  by  achieving  a  5  sortie  moving 
average  of  -3klbs.  This  is  an  example  of  a  possible  goal  for  the  experiment. 

4.  Impossible:  Attain  a  mean  score  in  the  20th  percentile  by  achieving  a  5  sortie  moving 
average  of  -4klbs. 

Several  implications  exist  within  this  goal.  First,  it  gives  the  AC’s  a  relative  standing 
amongst  their  peers.  Only  one  AC  out  of  24  scored  in  the  20th  percentile.  The  impact  of  not 
targeting  AC’s  in  the  highest  percentiles  is  negligible  compared  to  not  giving  the  majority  of 
AC’s  an  attainable  goal.  He  or  she  is  already  extremely  efficient  and  may  self  set  a  higher  goal. 
Next,  by  using  a  moving  average  of  sorties  in  the  experiment,  anomalies  can  be  mitigated.  If  an 
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AC  experiences  severe  weather  (and  ACFP  doesn’t  accurately  account  for  it),  the  rolling  average 
will  reduce  the  impact  and  eventually  fall  out  of  the  score.  Finally,  by  attaching  a  specific  klb 
figure  to  the  goal,  the  metric  becomes  very  important.  The  AC’s  must  trust  the  measurement  for 
the  assigned  klb  figure  to  have  any  motivational  impact.  Ml  cargo  Adjusted,  if  explained  properly  in 
the  commitment  attainment  phase,  will  have  this  buy-in. 

Future  Research 

More  research  is  needed  on  the  overall  performance  of  the  Fuel  Tracker  and  ACFP.  The 
underlying  assumptions  built  into  ACFP  should  be  validated.  For  instance,  what  impact  does 
ATC  allowed  hold-down  fuel  have  on  fuel  efficiency?  Is  added  fuel  for  extreme  weather 
accurately  allocated  in  fuel  efficiency  comparisons  and  calculations?  Should  a  cargo  adjustment 
be  built  into  ACFP?  How  can  the  system  be  leveraged  to  capture  important  facets  of  fuel 
efficiency  and  use  those  facets  to  impact  flying  behavior? 

Next,  the  culture  of  fuel  efficiency  in  the  Air  Force  should  be  evaluated.  Proper  metrics 
and  goals  can  have  a  significant  impact  but  the  underlying  culture  of  AC’s,  how  they  are  trained, 
what  is  deemed  important  to  them,  leadership  influence,  peer  influence,  and  other  factors  should 
be  studied  to  detennine  their  impact  on  fuel  efficiency.  This  focus  area  coupled  with  metric  and 
goal  development  could  prove  beneficial  to  the  Air  Force  in  the  future  of  fuel  efficiency  studies. 

As  previously  discussed,  the  selected  metric  and  fuel  goal  set  in  this  research  should  be 
utilized  in  a  motivational  study  within  Wings.  The  basic  premise  is  that  AC’s  will  be  given 
specific,  difficult  goals  based  on  the  metric.  Two  types  of  feedback  will  be  distributed  (public 
and  private)  to  the  AC’s  and  their  performance  will  be  evaluated  based  on  the  types  of  feedback. 
This  will  test  the  tenets  of  GST,  feedback,  and  metric  development  against  actual  perfonnance 
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and  help  detennine  if  a  behavioral  approach  to  fuel  efficiency  is  both  warranted  and  cost 
effective.  A  basic  model  of  this  future  research  can  be  found  in  Appendix  C. 
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Appendix  A:  C-17  Master  Fuel  Tracker  Worksheet 


C-17  Master  Fuel  Tracker  worksheet 


i 


POC  HQAMC/A3F 


(Mar  2014| 


Aircraft  Commander 


Squadron: 


Dept  Date  (Zuku): 


|  ICAO  From: 


Ramp  Fuel  (KB>s) 


Land  Fuel  (K)t>i'i 


Duration  (h.n| 


Cargo  *Pax  (Kltis) 


APU  (h.h) 


AJt  On  toad 


Did  you  tanker  fuel*  No 


Yes 


for  Cost  Avoidance 


Yes  forQperatiora  Requirement 


AmountTarkcred:  inos) 


Right  Plan  Used: 


ACF»  using  Ml  values 


AC=F  -sng  NA'  K'  vane; 


Was  Mi  F  Used: 

Yes 

No,  Flight  times  less  then  i  hour 
No  Sarte  3d  no t  sscve  lC.CCOn 


□  Ho.  Unapie  to  use  poth  MiF  atitudes  and  airspeeds 
O  No.  Due  to  slot  time  or  airfield  operating  hocrs 

□  Ho.  Due  to  crew  duty  day  restrictions 
•■3,  M  ssicr  QOmnuty  got  s.ai.asi* 


J  No,  Tanker  could  not  use  MIF  oue  to  Coronet 
:  No 


First  half  ERCC?  Yes  No  |  Hvy  Wt  A/R  Tracing*  Yes  No  |  Airdrop/Low  Level  Sortie:  Yes 


GPU  Used  Before  Takeoff: 

□  No,  GPU  Not  Funetiona 

Nc  Oserat  yisi  ■  Comeat.'ALStcre  Ocj! 


□Yes 

□No.  A/C  would  not  acceot  power 
No,  Mot  Required 


I  No,  Not  at  aircraft  oefdre  engine  start 
I  No.  Weather  precluded  use 


GPU  Used  At  Destmat»on : 

□  No.  GPU  Not  Funetiona 

No,  Operational  (Comtiat/Austere  Ops| 


□Yes 

□No.  A/C  would  rot  acceot  power 
i  No.  A/C  Requires  Tow  to  Parking 


.  No.  Not  at  aircraft  oefore  engine  start 
□  No.  Weather  precluded  use 
i  i  No.  Not  Required 


Ramp  Fuel  Deviation  Reason:  (Required  if  greater  than  1,500  LOs  difference  in  p«an  vs.  actual) 

□  N/A  J  Mission, '“Routing  mange 

□  Additional  Cargo/PAX  1 1  Enroute  WX 

□  AC  Adjusted  fuel  (FM  Agrees)  □  Fuel  Service  Over-Fuel 

□  AC  Adjusted  fuel  (FM  Disagrees)  (Please  Comment)  □  ERCC 


□  Tail  Swap/Couid  rot  defue 

□  Other  |  Please  Comment) 

C  Tankered  fuel1  (Ops/PM  Directed) 

C  Burred  ess  man  Plan  on  previous  eg 


Landing  Fuel  Deviation  Reason:  (Required  if  greater  man  3.000  LOs  difference  in  p«an  vs.  actual) 


□  N/A 

□  Airtiec  Ops 

□  ATC (Hoc  Downs.  Eicessive  vectors,  etc 

□  bash 

□  Few  less  than  scheduled  duraton-TNG  or  MSN  compete 

□  Air  Aoort:  Tanker  (Due  to  IFE.  Tanker,  MX  or  WX) 

Air  Aport:  Recever  iDue  to  s£,  Receiver  Vi  or  .vx; 


□  Mission  index  F*ying 

□  Cruise  Wind, 'Temp  Deviations 

□  Enroute  WX  Deviations 

□  Excessive  Fuel  Burr  in  Cruise 

□  Overfew  Original  Destination 

□  Extra  Cargo  .oaoed 

□  Ramp  Fuel  Deviation 


□  Maintenance 

C  Recovers  eie  not  show 

□  Receivers  took  i ess  fuel  man  panned 

□  Receivers  took  more  fue  than  plannee 

O  Atte  to  dimo  earlier  than  ACFP  Forecast 
C  Other _ 


Comments: 


I  wini 


Aircraft  Commander: 


Squadron: 


Dept  Date  (ZiWu): 


□ 


Ramp  Fuel  (1C os) 


Land  Fuel  (kids) 


Duration  (h.h| 


Cargo  *  Pax  (Kltis| 


APU  |h.h) 


(Kltisl 


d  you 


tanker  fuel* 


for  Cost  Avoidance 


Yes  for  Operational  Requirement 


Amount  Tankered:  |KlPs) 


Flight  Plan  Used: 


9  using  K 


'  ACFP  using  ~NA*  Mi  values 


Was  MiF  Used: 

□  Yes 

□  No.  Flight  times  less  man  l  hour 
Nc.  Ssrte  3 d  rot  ace-ate  aseve  1C  CCQ-. 


□  No.  Unaoie  to  use  poth  MiF  atitudes  and  airspeed: 

□  No.  Due  to  slot  time  or  airfield  operating  hours 

□  Ho.  Due  to  crew  duty  day  restriction: 

S3,  M  ssicr  compute*'  ’-op/rijt  a.aiiasie 


□  No.  barker  could  not  use  MIF  cue  to  Coronet 

□  NO 


Hvy  W*t  A/R  Training?  Yes  No  |  Airdrop/Low  Level  Sortie:  Yes 


First  half  ERCC?  Yes 


No 


GPU  Used  Before  Takeoff: 

□  No.  GPU  Not  Funetiona 

Nc  Oseraty.s'  Co-i cat.' Austere  Sc; 


□Yes 

□No.  A/C  would  not  acceot  power 
No.  Nat  Recjred 


□  No,  Not  at  aircraft  oefore  engine  start 
ONo.  Weather  precluded  use 


GPU  Used  At  Destination : 

□  No.  GPU  Not  Funetiona 

No,  Operational  (Com&at/Austere  Ops( 


□Yes 

□No.  A/C  would  not  acceot  sower 
No.  A/C  Requires  Tow  to  Parking 


1  No.  Not  at  aircraft  oefore  engine  start 
□No.  Weather  precluded  use 
:  :  No.  Not  Required 


Ramp  Fuel  Deviation  Reason:  (Required  rf  greater  than  1,500  Ltis  difference  in  plan  vs.  actual) 

□  N/A  □  Mission, '“Routing  mange 

□  Additional  Cargo/P  AX  □  Enroute  WX 

□  AC  Adjusted  fuel  (FM  Agrees)  □  Fuel  Service  Over-Fuel 

□  AC  Adjusted  fuel  (FM  Disagrees )  ( Please  Com  men?)  1  ERCC 


□  Tail  Swap/Couio  not  oeftie 

□  Other  please  Comment) 

C  Tankered  *u€f  (Ops/FM  Directed) 

C  Burrec  «ss  man  Plan  on  previous  leg 


Landing  Fuel  Deviation  Reason:  (Required  if  greater  man  3,0 

□  N/A 

□  Airftec  Ops 

□  ATC  (Hoc  Downs.  Eicessive  vectors,  etc 

□  BASH 

□  Flew  less  man  scheduled  duraton-TNG  or  MSN  compete 

□  Air  Aoort:  Tanker  (Due  to  IFE.  Tanker,  MX  or  WX) 

i  ]  Air  Aoort:  Receiver  (Due  to  if  E,  Receiver  MX,  or  WX) 


X)  Ltis  difference  in  plan  vs.  actual) 

□  Mission  Index  Flying 

□  Cruise  Wind, Temp  Deviations 

□  Enroute  WX  Deviations 

□  Excessive  Fuel  Burr  in  Cruise 

□  Overflew  Original  Destination 

□  Extra  Cargo  .oaoed 

□  Ramp  Fuel  Deviation 


□  Maintenance 

C  Receivers  did  rot  show 

□  Receivers  took  less  fuel  man  ponned 

□  Receivers  took  more  fue  mar  planned 

□  Ati<e  to  camo  eerier  man  ACFP  Forecast 

C  Other _ 


Comments: 
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Appendix  B:  Fuel  Efficiency  Background  Metric  (Reiman,  2014:  51-52) 


6  —  /?0  +  filOC  +  /?2^2  4~  /) 3^0  4  /?4<x)2  4"  fi^ccOO 


(21) 


Where: 

9  =  Specific  Range  in  NMs  per  Klbs 

a  =  Altitude  in  Thousands  of  Feet 

co  =  Aircraft  Gross  Weight  in  Klbs 


Table  3:  Specific  range  regression  terms 


C-5 

C-17 

C-130 

Po 

24.538 

31.735 

58.829 

A 

0.5511 

0.9897 

3.5292 

P2 

0.0002 

-0.0043 

-0.0098 

P3 

-0.0318 

-0.0642 

-0.2384 

P* 

1.9E-05 

5.8E-05 

0.0010 

Ps 

-0.0005 

-0.0011 

-0.0155 

Given  the  specific  range  regression  equation,  the  distance  flown  in  NMs  for  a  given 
altitude  and  gross  weight  can  be  determined  by  integrating  Equation  2 1  with  respect  to  the 
change  in  fuel  consumed  over  the  interval  from  zero  to  the  total  fuel  consumed  as  shown  in 
Equation  22.  After  integrating  and  solving  for  cruise  fuel,  the  resulting  equation  is  as  shown  in 


Equation  23. 


S  —  f0  ff(P0  +  Pia  +  /?2«2  +  P 3 +  Pa^2  +  PsaC0)  df 


(22) 


B  13  1 


■ f  3,4  3A  J  2 


2B3  -  9 ABC  +  27 A2D  +  V(2B3  -  9 ABC  +  27 A2D)2  -  4 (B2  -  3AC)3] 
-  ^  [2B3  -  9 ABC  +  27 A2 D  -  7(2 B3  -  9 ABC  +  27 A2D)2  -  4 (B2  -  3 AC)3]  (23) 


Where  (All  weights  in  Klbs): 

A 

3 

B  =  (y  +  Pa(^oP  +  w/rc  +  Mfah  +  wp)  +  y  a) 

c  =  Po  +  Pla  +  p2a2  +  /?3(o)op  +  w/rc  +  Mfah  +  top)  + 

2 

Pa^S^op  3~  ^frc  "h  (tifah  3”  top)  3"  Ps^i^^op  3”  to frc  3”  to fah  3”  top) 
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D  =-S 

S  =  Distance  in  NMs 

a  =  Altitude  in  Thousands  of  Feet 

oo  =  Aircraft  Gross  Weight 

=  Wop  +  w frc  +  W fah  +  W  p+f 

co  op  =  Operating  Weight 

co frc  =  Reserve/Contingency  Fuel  Weight 

co fah  =  Alternate/Holding  Fuel  Weight 

c op  =  Payload  Weight 

/  =  Fuel  Consumed 

c Off  =  Cruise  Fuel  Weight 
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Appendix  C:  Bridging  the  Research  (Follow-On  Experiment) 
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Appendix  D:  Mean  Efficiency  Scores  Sepll-Augl4 
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Appendix  E:  FCIF  Changes  Jun  11  -  Dec  14 


FCIF# 

Release  Date 

Subject 

11-06-04 

14- Jun- 11 

Pitot’s  Performance  Advisory  System  (PPAS)  Operational  Procedures  ibr  C- 1 7  Aircraft 

11-06-06 

17- Jun- 11 

Destination  Weather  Requirements 

11-07-07 

19- Jun- 11 

Equal  Time  Point  and  Depressurization/Decompression  Fuel  Procedures  Update 

11-09-06 

22- Sep- 11 

Early  Descent  Fuel  Calculation  ibr  KC-  10s 

11-10-11 

24-Oct-l  1 

Early  Descent  Fuel  Calculation  for  all  other  MDSs 

12-07-02 

9-Jul-12 

Cost  Avoidance  Tankering  (MAFCAT)  Program 

12-08-10 

28-Aug-12 

Cargo  Loading  for  Optimal  Fuel  Efficient  Center  of  Gravity 

13-02-11 

22-Feb-13 

Fuel  Tracker 

13-07-07 

26- Jul- 13 

C- 1 7A  Tail  Specific  Fuel  Bias  Checklist 

13-07-09 

30- Jul- 13 

Waiver  to  OCONUS  Alternate  Requirements 

14-01-03 

10- Jan- 14 

Fuel  Tracker 

14-10-17 

20-Oct-14 

C-17  Authorized  Identified  Extra  Fuel 

14-12-02 

3-Dec- 14 

Flight  Planning  Fuel  Policy  Letter 
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Appendix  F :  M2RegreSsion  Multivariate  Plot 


A  Multivariate 
a  Correlations 


Cargo(Cargo  Act+(.5*OffloadyOnload)  klbs  Cargo  Act*2  Fly  Time  Act  Fly  Time  ActA2Cargo*Fly  Time  Actual  Fuel  Consumed 


Cargo(Cargo  Act+(.5*Offload/Onload)  klbs 

1.0000 

0.9410 

0.2955 

0.2472 

0.7756 

0.3707 

Cargo  ActA2 

0.9410 

1.0000 

0.2175 

0.1703 

0.7301 

0.2926 

Fly  Time  Act 

0.2955 

0.2175 

1.0000 

0.9641 

0.6780 

0.9884 

Fly  Time  Act*2 

0.2472 

0.1703 

0.9641 

1.0000 

0.6437 

0.9567 

Cargo*Fly  Time 

0.7756 

0.7301 

0.6780 

0.6437 

1.0000 

0.7458 

Actual  Fuel  Consumed 

0.3707 

0.2926 

0.9884 

0.9567 

0.7458 

1.0000 

-d  -  Scatterplot Matrix 


■  i 1 1  ■  i  ■  i  •  i  ■  i '  i '  i '  i 1 1 1 1  ■  i  ■  i  •  WTwmwmwr — v  m  n  n  n  n  i1 11  p  n  p  n  n  ^rrFFFrrrrFFFFFrrr  wtwtwtwtwtiwi  wrwiwTwrwnwrwi 

0  20  40  60  80  100  0  3000  700011000  0  1  2  34  5  67  8  9  11  0  20  50  80  110 140  0  150  350  550  750  0  30  60  90120  160  200 
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Appendix  G:  Fuel  Planning  Policy  letter  Nov  14  (Gillson,  2014) 


Flight  Planning  Fuel  Policy  Letter  14.2 

Additional  Fuel/Default  Mission  Index  (MI): 


Due  to  international  airspace  inconsistencies  (erratic  ATC  clearances,  ADS-B  forced  descents)  and  flight 
planning  software  constraints,  planners  are  directed  to  add  the  following  additional  contingency  fuels  to  the 
ACFP  Flight  Plan  Request.  MI  values  listed  below  are  ACFP  defaults: 


Aircraft 

Hold  Down/Earlv  Descent 

Egvpt  OfF 

Singapore  O/F1 

MI 

C-130 

N/A 

2,500 

N/A 

N/A 

C-17 

4,500/4,500 

8,000 

4,000 

31 

C-5 

6.000  6.000 

12,000 

6.500 

41 

KC-10 

4.500  4.500 

12,000 

5.000 

34 

KC-135 

3.000  3.000 

8.000 

4,000 

R-24.  T-19 

C-21 

200  200 

NA 

N/A 

N/A 

1  Do  not  indud 

e  Singapore  O  F  fuel  when  WSAP  is  the  departure  or  arrival  ICAO. 

Planners  will  use  “Fuel  D”  for  Hold  Down  fuel,  and  “Fuel  I”  for  Early  Descent  Fuel  and  Overflight  (O  F)  Fuel. 
Planners  null  not  bump  cargo  for  additional  O  F  Fuel  or  apply  O  F  fuel  additions  to  active  CORONET  legs.  Do 
not  add  Hold  Down  Early  Descent  fuel  when  sortie  is  planned  under  FL180. 


Hold  Down/Early  Descent  List: 


KADW1 

KDOV12 

KMRB1-2 

KSWF1 

KPIT3 

KCEF1 

KWRI1 

WSAP 

RJTY 

RJTA 

OAKN 

1  ForC-21.  only  applies  to  arrivals  from  the  west 
2ForC-5.  only  applies  to  arrivals  from  the  west 

1  Only  applies  to  flights  transiting  to  from  the  east  to  the  northeast  (KADW  north  through  New  York  ARTCC  airspace). 

CORONET :  Tankers  may  carry'  up  to  1  hour  extra  holding  fuel  on  active  AR  legs. 

ACFP:  All  C- 5s  have  a  4%  degrade  automatically  added  by  default  UFN. 

AOR/C  ombat  Zone : 

-Planners  will  tanker  as  much  fuel  as  practical  into  Afghanistan  and  always  tanker  through  Iraq.  Otherwise, 
tanker  fuel  IAW  MAFCAT  Fuel  Tankering  Matrix  located  on  61 S  AOC  (TACC)  website. 

—Planners  will  not  bump  cargo  or  passengers  to  tanker  fuel  if  fuel  is  available. 

—Operational  requirements  take  precedence  over  fuel  tankering  policy. 

— OASH:  Calculate  an  SDP  to  depart  OASH.  If  planning  to  tanker  fuel  over  the  value  needed  to  get  to  the  next 
refueling  location,  limit  the  TOGW  to  SDP  weight  minus  25K  (do  not  reduce  ACL  in  order  to  increase  tankered 
fuel,  this  does  not  preclude  going  up  to  full  SDP  TOGW  to  get  to  the  next  ICAO). 

-Planners  will  include  a  destination  alternate  when  departing  a  location  with  limited  fuel. 

-Planners  will  plan  an  alternate  for  anv  mission  into  an  AOR  Combat  zone. 

-OAIXMin  Landing  Fuel  for  C- 17s  &  Cos  is  2  IK  (3K  extra). 

NOTE:  The  Aircraft  Commander  (AC)  is  the  final  authority'  regarding  aircraft  fuel  loads.  Planners  will  comply 
with  AC  direction. 


POC:  HQAMC/A3V,  Mr.  Andrew  B.  Gillson,  DSN  779-3626  Latest  Update:  17November2014 
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Appendix  H:  Storyboard 


ACTIVE  DUTY  C-17  AIRCRAFT  COMMANDER  FUEL 
EFFICIENCY  METRICS  AND  GOAL  EVALUATION 


a 


Introduction  Metrics 

Optimal  metric*  and  goals  are  essential  componettsto  encourage 
efficient  C- 1 7  flying  Hits  research  examined  two  metrics  ihmnjh 
quantitative  statistical  and  qualitative  criteria  analysis  picked  the  most 
effective  metric  and  utdued  Coal  Setting  Theory  (GST)  to  couple  the 
metric  with  an  attainable  goal  aimed  at  making  Aircraft  Commander  » 
(AC*)  more  fuel  efficient  The  first  metric,  M 1  {*,.*<««  “***  planned 
fuel  use .  calculated  in  the  Advanced  Computer  Flight  Plan  (ACFP),  and 
adjust*  forpayfoad  The  second  metric,  NCn^._.  use*  repessice 
analysis haird  on  fitful  time  and  carp)  to  drtentane  predicted  sortie  fuel 


C  ap!  Michael  J.  Schumacher 
Advisors:  I.t  Col  Adam  Reiman 
Dr.  Ken  Schultz 

Department  of  Operational  Sciences  (ENS) 


Results 

IQ  1  Thesis  attributes  were  quantitative!  v  scored  in  decision  matrices  and  qualitatively  analyzed 
based  on  metric  criteria  attittte*  Quantise  vrly  M 1  cqi  u.m  k<*  ed  higher  in  each  of  the  ulilued 
matrices  Qaahutivdy,  stark  differences  existed  between  the  metrics  regnding  how  ACFP  planned 
and  repesixm  ptcdelrd  fuel  valoes  could  be  in  fl  wooed  by  AC  s  ACFPplanaed  values  suet  as 

(regression  variables)  exhibit  tesscontrolUW*  faclots  lo  the  AC  * 

IQ.'  Much  greater  variability  and  a  less  efficient  mean  efleirory  score  existed  with  ii 
M 1  demonrtued  that  fuel  efficiency  using ACTTplanning  factors  thorn  more  efficient 

sorties  than  recession  analysis  would  suggest 


Problem  Statement 

EnsuringAF  AC's  arr  focused  on  fuel  efficiency  is  paramcintf  to 
sustaining  teioiscei  and  reducing  costs  Cuncotlv  fuel  efficiency 
metrics  are  not  utilized  lo  inlkioice behavior  while  flying  Without  a 
well  defined  metric  tit  place  fot  efficiency,  goals  cannot  be  set  or 
attained  the  researcher  will  compare  two  efficiency  metrics,  select  tbe 
most  useful  one  and  use  that  metrv  to  set  an  efficiency  goal  for  all  AF 
AC’s  This  goal  will  be  the  cornerstone  for  a  follow  on  experiment 
assessing  the  effect  of  publicly  or  privately  provided  feedback  on  AC's 
fuel  efficiency 

Investigative  Questions 

to  1  From  a  metric crilrria perspective,  is  Ml.  or  . i 

more  adequate  measure  of  fuel  efficiency  fot  AC  s'* 


SI® 


102  From  a  metric  comparison  perspective  ,  how  do  Wingi  Imkndual 
AC’s  perform” 

IQ?  Which  metric  should  be  provided  lo  AC  *  and  fot  what  purpose? 

1Q«  Using  goal  theory  literature  how  can  tbe  chosen  metric  be 
presented  as  a  goal  for  AC’s?  What  efficiency  goal  can  be  set  for  AT 
AC’S? 


Conclusions 

M 1  provider  a  locus  of  control  to  AC  i  which  will  influence 

their  belief  in  the  ability  to  influence  the  metric  and  fly  more  efficiently 

•ACFP  factors  such  as  planned  wind,  planned  tempnrUure  and  planned 
altitude  could  be  influenced  under  Ml  tn  methods  that  the 

regress*®  variables  could  not 

•  The  set  goal  in  IQ4  provide*  a  robusr  measurement  of  AC  perfomunre 
for  seven!  reasons 

•  The  relative  standing  (percentile)  provides  tbe  AC  a  performance 
level  in  relation  to  thee  peers 

•  The  moving  average  help  to  nutigatr  poor  perfomuareoo  a 
single  sortie. 

•  Using  exact  klbs  fuel  as  a  final  amottd  for  AC's  to  strive  for 
meet*  GST  tenet*  of  specific  Ky  and  difficulty 

•AMO AT  F  Fuel  Efficiency  Initiatives  are  hiving  a  great  impart  on  fuel 
effieimey  Using  ACFP  planning  factors  inetxpcnffd  tnto  SI  1  ^ 
aS19  1 M  cost  swing  has  occwed  from  2011  to  2014 


SQ  Qavwi*  M  ET&-W.  biW  hpll  » 


Sponsor:  AMC/A3F 


March  2015 


Future  Research 

•  Advanced  Computrt  Fli^il  Plan  (ACFP)  Perform**^-  ACFPujidetlyng 
formulatiom  should  be  reexamined  lo  determine  if  they  adequately 
account  fot  fuel  usage  during  sortie  planning 

•  Culture  of  Fuel  Efficiency  Thu  research  showed  that  AC  s  performed 
drastically  different  Follow-on  research  shcnld  study  why  this  is  the 
case  Why  are  some  AC  s  efficient  fivers  while  othos  are  inefficient” 

•  Motivations!  Study  WSth  Feedback  With  the  mrtric  selected  and  the 
effieimey  goal  in  plare  a  motivational  study  it  warranted  to  determine 
their  effects  In  this  study  the  metric  and  goal  would  be  presetted  to 
AC's  with  multiple  forms  of  feeAark,  tnchidmg  public  and  private,  to 
determine  if  GST  coupled  with  fee&ack  influence*  performance 


Department  of  Operational  Sciences 
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